openLuup vs Vera - comms protocol

Plugins & Plugin Development openLuup
1

A few notes on the comms protocol implementation - these are just observations and may be proved to be false. Any comments welcome:

  1. With no tags in either of the I_xxxx file or the D_xxxx file, then the Vera 3 default protocol is crlf - as determined by WireShark.

  2. Vera 3 sets the file /etc/sysctl.conf to contain:

net.ipv4.tcp_keepalive_time=120

The usual defaults are:

tcp_keepalive_time = 7200 (seconds) tcp_keepalive_intvl = 75 (seconds) tcp_keepalive_probes = 9 (number of probes)

It would seem that Vera 3 starts using keepalives very soon in the piece: 2 minutes vs 2 hours. This may affect keepalive compatibility depending on the OS defaults, connected devices in use, routers, etc. It would seem that some slave devices source their own keepalives. eg GC100 produces one every 10 seconds.

  1. luup.io.intercept()

Testing suggests the following:

a) Of note: luup.io.intercept() returns a boolean
b) on entry to any function, an initial call to luup.io.intercept() returns true
c) a call to luup.io.write() in that function results in a subsequent call to luup.io.intercept() returning false
d) any number of subsequent calls to luup.io.read() after that initial luup.io.write() in that function makes no difference in what luup.io.intercept() returns. It will continue to return false in all cases.
e) on exit of the function the luup.io.intercept() “appears” to be always (re)set to true, ready for any new function that may call luup.io.intercept()
f) may be passed a device ID parameter but without one, defaults to the current device.

How would that be implememted?

2

OK, thanks for the experimentation! I generally aim to match Vera functionality, but if it is not documented (and, come to think of it, lots of things aren’t!) there’s a limit to how accurate the emulation can or should be. The behaviour may be accidental anyway.

1) With no tags in either of the I_xxxx file or the D_xxxx file, then the Vera 3 default protocol is crlf - as determined by WireShark.

This seems clear and straight-forward, so I can see about including it. Any decent plugin author should specify this, but I guess you have found an historical case where this wasn’t done.

2) Vera 3 sets the file /etc/sysctl.conf to contain...

The openLuup installation does nothing to system files and almost nothing outside its own directory tree (a symbolic link to /var/log/cmh/ is perhaps the exception.)

If you need to change the system behaviour, for example to make openLuup start up at boot time, then it’s up to you!

The io.lua module does enable its own socket keepalive, but I don’t know what time interval it uses, or indeed how to change it.

3) luup.io.intercept()

Testing suggests the following:

a) Of note: luup.io.intercept() returns a boolean
b) on entry to any function, an initial call to luup.io.intercept() returns true
c) a call to luup.io.write() in that function results in a subsequent call to luup.io.intercept() returning false
d) any number of subsequent calls to luup.io.read() after that initial luup.io.write() in that function makes no difference in what luup.io.intercept() returns. It will continue to return false in all cases.
e) on exit of the function the luup.io.intercept() “appears” to be always (re)set to true, ready for any new function that may call luup.io.intercept()
f) may be passed a device ID parameter but without one, defaults to the current device.

How would that be implememted?

…WHY would that be implemented? The boolean return is complex, undocumented behaviour, and caveat implementor anyone who tried to make good use of it. What would it be used for?

The crucial thing here is what intercept() actually does, as you described previously, which is closer to the way that I thought it should work than is currently implemented. AFAIK, the only other people brave enough to dabble in this area are @cybrmage who did some extensive testing on a particular plugin, and @martynwendon who tested the keepalive feature for me. It would be nice if we could agree on the functionality to a point where it worked for all of you.

However, hats off to your extensive testing, which I much appreciate and will investigate further. The short-term question is what you might need changed just to get something working for now.

3
1) the Vera 3 default protocol is crlf - as determined by WireShark.
Yes an easy one - it's got to default to something and crlf makes sense.
2) Vera 3 sets the file /etc/sysctl.conf to contain..
Just a warning to others, that it may make a difference.

I have one plugin where the socket that uses intercept() appears to close after about 25 minutes unless I give it a poke. I’m still working on exactly how long and why. Another plugin I have that solely uses has worked flawlessly for days with openLuup.

intercept()

Yes - forum members are rather suspicious of intercept(). I’ve no idea why it returns a boolean or why you would want to pass a device number to it either - probably just to conform to how other stuff is done.

I had a horrendous time with the Arduino Sensors plugin clashing with my “processIncoming” function using the tag. Took me a long time to figure that one out and now have only half my hair. So I got sucked into using iointercept() especially as these other plugins were just a send and then receive something back in response. But there seems to be this ongoing fear that something will slip through to the tag - I’ve had no trouble.

Futzle probably had the right idea when she said (and I hopefully interpret her words correctly) abandon io.intercept() and use a state machine on the tag and treat all data as asynchronous. I might rewrite my plugins that use intercept to do that instead.

The documentation implies you do the following. openLuup disables the intercept after each read, so this works OK:

intercept
write
read

intercept
write
read

intercept
write
read

But what about if using the raw protocol where just one byte is returned per read. The overhead is considerable. I have done the following with Vera and it works fine. But openLuup doesn’t allow it (currently):

intercept
write
read
read
read
read
read
read
read
read
read
read
etc

Watching the boolean that luup.io.intercept() returns may well be informative as a way of determining Vera’s interception implementation. I’m guessing but it sort of looks like the function that contained the luup.io.intercept() call automagically turns the intercept back off when the containing function is exited/destroyed. I might be imagining it but like so:

local function myGreatCommsFunction()
   intercept
   write
   read   -- intercept in place
   read   -- intercept in place
   read   -- intercept in place

end    -- intercept disabled

intercept may also return a different values depending on whether lup.io.open of the ip address has been completed or not at start up.

In summary it’s clear luup.io.intercept() lasts more than one read. There is still room for more testing.

4

Interesting (again.)

What if intercept was only cancelled after a read when using non-raw protocol?

5
What if intercept was only cancelled after a read when using non-raw protocol?

No - io.intercept seems to work over multiple reads using raw. Interspersing io.intercept boolean checks during consecutive reads seems to indicate the intercept is still in place.

6

[quote=“a-lurker, post:5, topic:196129”]

What if intercept was only cancelled after a read when using non-raw protocol?

No - io.intercept seems to work over multiple reads using raw. Interspersing io.intercept boolean checks during consecutive reads seems to indicate the intercept is till in place.[/quote]

No - let me put it another way: only allow io.intercept to persist WITH multiple reads using raw.

7

Yes - that sounds good.

8

Development branch updated (2017.04.27) to include this luup.io.intercept() modification.