I’m writing my first Vera plugin and I’m now trying to understand what protocol I’ll use.
I’m trying to receive a HTTP request, so I added some code in my implementation tag and what I noticed is this:
With protocol crlf: the input to my code (lul_data) is a full line of data with the CR/LF stripped (as expected). If the other end of the connection sends me data that’s not terminated by CR/LF, Luup will cache it.
With protocol raw: the input to my code (lul_data) is one character at a time and it will include any line endings present in the data stream (as expected). However, here’s the part I’m having trouble with: if the other end of the connection sends me data not terminated by CR/LF, Luup is apparently caching the data (even though I have my protocol set to raw) and is not passing it to my code until I artificially add a CR/LF at the end of the stream. When I do, then I get the same one-byte-at-a-time delivery of the data, against containing the CR/LF at the end.
Basically what I’m trying to do is read a fixed number of bytes (provided by the Content-Length header) and the request body is around 3,500 bytes long and happens to not contain a single CR/LF in it.
How does one convince Luup to deliver data without CR/LF?
If you’re down at the level of coding TCP I/O with sockets and the tag, then you’ll have to handle all the HTTP protocol of headers and responses yourself. Much harder.
I had looked into a luup callback handlers and while they make things much easier, their limitations basically make a solution impossible in my case as I need to be able to handle arbitrary URLs. That’s why I went down the path I am now. Everything mostly works, but I just hit this issue and I’m curious about how other people are handling it.
Mind you, it happens to be applicable to HTTP in my case, but I can see this happening with any arbitrary protocol that happens to not send CR/LF often or at all during transmission.
In that case, you’d almost certainly be better off writing your own handler from scratch. You can do this directly in Lua using the LuaSocket library. Several functions in that library help with doing a non-blocking poll of the socket to see if there is data to read.
My experience of trying to reverse-engineer the luup.io module shows that this is a deeply flawed implementation.
Got it. My current solution kind of handles all the socket level traffic on its own, but I went out of my way to try to make things play nicely with Luup. Too bad Luup is not playing nicely with me.
What are the rules of the game If I were to do everything on my own? I supposed I shouldn’t have any blocking code in any of my luup callbacks (e.g.: startup, incoming, etc), so I could get everything started in my startup function, but I’m not sure where I would put my loop to select() on the socket I have.
If I’m not going to use luup.io, then my incoming callback will never be called. Is there a callback that I can add and have Luup call me back on a regular basis so I can check for I/O?
There’s three examples that I could point to, although I wouldn’t necessarily describe them as best practice, they do work. One of them is almost entirely someone else’s work:
[ul][li]Alternate Event Server - used in the EventWatcher plugin for UI5 (UI7 broke this capability)[/li]
[li]DataYours - the UDP receiver in the DataWatcher daemon[/li]
[li]openLuup - pure Lua, but emulates the MiOS system, rather than runs under it. The scheduler is key[/li][/ul]
My experience with the RAW protocol (the Caddx alarm plugin) does not mirror the behaviour you are describing. Each byte arrives immediately without any line buffering. Granted, that plugin is using a serial port rather than a network connection but I’d be surprised if that was a factor, given how Vera serial ports are fed through a ser2net process anyway.
This is great, thank you very much for showing me the way here. It looks like the luup.call_delay() function is the basis of it all. I’ll look into it.
[quote=“futzle, post:7, topic:198538”]My experience with the RAW protocol (the Caddx alarm plugin) does not mirror the behaviour you are describing. Each byte arrives immediately without any line buffering. Granted, that plugin is using a serial port rather than a network connection but I’d be surprised if that was a factor, given how Vera serial ports are fed through a ser2net process anyway.
A thought: are you sure the other end of your network connection isn’t buffering its output?[/quote]
Thank you for chiming in, futzle. Good to know of your counter-example. I’ll check your code.
Since I’m testing things right now I have full control over the connection, both sides. Having said that, it’s entirely possible I messed up somewhere. Given what you said, I’m encouraged to go take another, more careful look.
[quote=“darthray, post:9, topic:198538”][quote=“futzle, post:7, topic:198538”]My experience with the RAW protocol (the Caddx alarm plugin) does not mirror the behaviour you are describing. Each byte arrives immediately without any line buffering. Granted, that plugin is using a serial port rather than a network connection but I’d be surprised if that was a factor, given how Vera serial ports are fed through a ser2net process anyway.
A thought: are you sure the other end of your network connection isn’t buffering its output?[/quote]
Thank you for chiming in, futzle. Good to know of your counter-example. I’ll check your code.
Since I’m testing things right now I have full control over the connection, both sides. Having said that, it’s entirely possible I messed up somewhere. Given what you said, I’m encouraged to go take another, more careful look.
Thank you![/quote]
Just closing the loop on this, futzle, you were right. Things are working just fine. As you suspected, the client side of the connection was only sending data on end-of-line. I feel silly. :-[ Thanks for calling that out.
Best Home Automation shopping experience. Shop at Ezlo!
