openLuup: Ademco Vista Alarm Panel with EVL3

akbooer · January 26, 2016, 10:19am

Continuing the discussion from here: http://forum.micasaverde.com/index.php/topic,35972.0.html

CudaNet · January 26, 2016, 6:08pm

Always here to help so please let me know what you need.

akbooer · January 26, 2016, 9:39pm

I’m currently testing (to the best of my ability) modifications which implement I/O intercepts incorporating the new understanding from @cybrmage. So I’ll have something to try tomorrow, I hope. The only hardware I have which uses I/O (but not intercepts) is the MySensors plugin, but that’s not a good test - I’ll try a special test harness tomorrow.

CudaNet · January 27, 2016, 12:07am

Sounds like a plan. One you feel comfortable releasing it in the “wild”, I can give it a quick go and report back.

I’m currently testing (to the best of my ability) modifications which implement I/O intercepts incorporating the new understanding from @cybrmage. So I’ll have something to try tomorrow, I hope. The only hardware I have which uses I/O (but not intercepts) is the MySensors plugin, but that’s not a good test - I’ll try a special test harness tomorrow.[/quote]

cybrmage · January 27, 2016, 1:26am

Just to give more info on how Vera IO works…

The IO protocol is defined in the device xml file… It can be “raw”, “lf” or “crlf”

In “lf” and “crlf” mode, incoming data is passed to the function line at a time, with the terminating “lf” or “crlf” removed. Calls to luup.io.read return the same. Calls to luup.io.write send the data with the appropriate terminator appended to the passed data.

In “raw” mode, incoming data is passed to the function (or to luup.io.read) one byte at a time, with no data modification. Calls to luup.io.write sends the passed data in it’s entirety, with no terminators added.

Regardless of the selected mode, calls to luup.io.intercept are terminated after the next call to luup.io.read.

As far as lua socket io is concerned, “lf” and “crlf” modes would use a socket:read(“*l”) call and raw mode would use a socket:read(1) call.

As far as my plugins are concerned:

Caseta Connect: uses raw mode for communication with a SmartBridge pro and with the Lutron MQTT broker. This is required due to the binary data format used by MQTT, and the inability to easily change the protocol on the fly. The plugin has been updated so that it detects when it is not running on Vera hardware and compensates for the differences in platforms (ie: luup io differences and dropbear/openssh on other platforms)

EVL3Vista: Uses “crlf” mode and works properly once manually configured. The configuration helper functions (read panel, read labels, set labels and usercode management) all make extensive use of the luup.io.intercept functionality, and do not function properly.

Wink Connect: Uses the luup.io subsystem for PUBNUB notifications (which is an http protocol) and, except for v0.21rc1 (which imbeds it’s device files into the plugin and writes the out to the filesystem as lzo files), should function normally. A minor modification to the plugin should resolve the issues with v0.21rc1.

akbooer · January 27, 2016, 10:42am

[quote=“cybrmage, post:5, topic:190783”]Just to give more info on how Vera IO works…
[…]
As far as lua socket io is concerned, “lf” and “crlf” modes would use a socket:read(“*l”) call and raw mode would use a socket:read(1) call.[/quote]

Thanks for that clarification - which now matches my own understanding. In fact, the changes I am already testing do exactly that. I have not bothered to implement ‘stxetx mode’ (which may be needed in future for serial hardware?)

My source of previous misunderstanding was mainly the belief that io.intercept() was a one-time only thing, rather than a per-read one. I note that the protocol may also be defined in the implementation file.

As far as my plugins are concerned: [...]

So raw mode should shortly work, but note that openLuup does not use .lzo compression so there may be some incompatibility there still?

Anyway, hope to see this all up and running soon!

Thanks again for the comprehensive feedback.

akbooer · January 27, 2016, 6:08pm

Here’s some replacement modules which implement the changes for luup.io as described. You may care to back up your current configuration!

I’ve tested this with a test harness, toggling intercepts on and off and mixing I/O handled with read/write and the incoming handler. It also implements the protocols “cr”, “crlf”, and “raw”, but not “stxetx”.

All seems OK to me.

Does this float the EVL3 boat?

CudaNet · January 27, 2016, 6:14pm

I’ll run this 1st thing tonight unless Cybrmage has an opportunity.

Here’s some replacement modules which implement the changes for luup.io as described. You may care to back up your current configuration!

I’ve tested this with a test harness, toggling intercepts on and off and mixing I/O handled with read/write and the incoming handler. It also implements the protocols “cr”, “crlf”, and “raw”, but not “stxetx”.

All seems OK to me.

Does this float the EVL3 boat?[/quote]

CudaNet · January 27, 2016, 10:16pm

Loaded the updated files.

-rw-r--r--    1 root     root         11684 Nov 17 07:42 chdev.lua
-rw-r--r--    1 root     root         11926 Nov 17 07:42 devices.lua
-rw-r--r--    1 root     root          8108 Nov 17 07:42 gateway.lua
-rw-r--r--    1 root     root          8311 Nov 17 07:42 init.lua
-rw-r--r--    1 root     root         11480 Jan 27 11:00 io.lua
-rw-r--r--    1 root     root         11000 Nov 17 07:42 json.lua
-rw-r--r--    1 root     root         15278 Jan 26 22:40 loader.lua
-rw-r--r--    1 root     root          8966 Nov 17 07:42 logs.lua
-rw-r--r--    1 root     root         21979 Nov 17 07:42 luup.lua
-rw-r--r--    1 root     root          8241 Nov 17 07:42 plugins.lua
-rw-r--r--    1 root     root         22132 Nov 17 07:42 requests.lua
-rw-r--r--    1 root     root          2964 Nov 17 07:42 rooms.lua
-rw-r--r--    1 root     root          7096 Nov 17 07:42 scenes.lua
-rw-r--r--    1 root     root         16646 Jan 26 22:47 scheduler.lua
-rw-r--r--    1 root     root         14706 Nov 17 07:42 server.lua
-rw-r--r--    1 root     root         13746 Nov 17 07:42 timers.lua
-rw-r--r--    1 root     root          5602 Nov 17 07:42 userdata.lua
-rw-r--r--    1 root     root          4257 Nov 17 07:42 xml.lua

Started with a new user_data.json.
Executed openLuup.
Enabled AltUI debug mode.
Installed EVL3 Ademco plugin.

luup.create_device ('', "Vista20p", "Vista20p", "D_EVL3VistaAlarmPanel1.xml")

Selected Vista Plus Series:20p [Panel].
Changed Installer Code from ‘4112’ to ‘XXXX’ [Codes].
Added Installer Code ‘XXXX’ [Panel].
Added Interface Password:user [EVL3Vista].
A. Pressed the [Reload Engine] button.

Wed Jan 27 16:05:56 2016        device 0 '_system_' requesting reload

B. Pressed ‘Detect’ button [EVL3Vista]. I confirmed IP and Version were indeed detected. Plugin Status: No Zones Defined.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2438    0  2438    0     0   120k      0 --:--:-- --:--:-- --:--:--  132k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5086    0  5086    0     0   101k      0 --:--:-- --:--:-- --:--:--  103k

C. Pressed ‘Read Panel Config’ [Panel]. Plugin Status : Busy. Keypad chimed one time.
D. Appears to have stopped (see logs):

Excerpt of errors (I may have missed others):

[code]
2016-01-27 16:05:57.095 openLuup.context_switch:: ERROR: [string “[4] I_EVL3VistaAlarmPanel1.xml”]:8719: attempt to concatenate local ‘isDisabled’ (a nil value)
2016-01-27 16:05:57.096 openLuup.scheduler:: job aborted : [string “[4] I_EVL3VistaAlarmPanel1.xml”]:8719: attempt to concatenate local ‘isDisabled’ (a nil value)

2016-01-27 16:10:50.886 luup_log:4: (EVL3VistaAlarmPanel::discoverPanelConfig) Attempting to get zone data…
2016-01-27 16:10:50.887 luup_log:4: (EVL3VistaAlarmPanel::discoverPanelConfig) Panel: Type [NIL] model [NIL].
2016-01-27 16:10:50.888 luup.task:4: status=2 EVL3VistaAlarmPanel : Attempting to get zone data…
2016-01-27 16:10:50.888 luup.variable_set:4: 4.urn:micasaverde-com:serviceId:EVL3VistaAlarmPanel1.Plugin_Status was: No ZONES defined… now: BUSY #hooks:0
2016-01-27 16:10:50.890 luup_log:4: (EVL3VistaAlarmPanel::discoverPanelConfig) Testing installer code.
2016-01-27 16:10:51.141 luup.io.read:: error: timeout
2016-01-27 16:10:51.391 luup.io.read:: error: timeout
2016-01-27 16:10:51.642 luup.io.read:: error: timeout
2016-01-27 16:10:51.893 luup.io.read:: error: timeout
2016-01-27 16:10:52.144 luup.io.read:: error: timeout
2016-01-27 16:10:52.394 luup.io.read:: error: timeout
2016-01-27 16:10:52.645 luup.io.read:: error: timeout
2016-01-27 16:10:52.646 luup.io.write:: message length: 7, bytes sent: 7, status: OK
2016-01-27 16:10:52.647 luup.io.read:: error: timeout
2016-01-27 16:10:52.648 openLuup.context_switch:: ERROR: [string “[4] I_EVL3VistaAlarmPanel1.xml”]:7325: attempt to compare number with nil
2016-01-27 16:10:52.648 luup.delay_callback:: function: 0x213d478 ERROR: [string “[4] I_EVL3VistaAlarmPanel1.xml”]:7325: attempt to compare number with nil
2016-01-27 16:10:52.649 luup.io.incoming:: bytes received: 37, status: OK
2016-01-27 16:10:52.657 openLuup.server:: request completed (2072 bytes, 1 chunks, 30051 ms) tcp{client}: 0x216ed08
2016-01-27 16:10:52.658 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:10:52.792 openLuup.server:: /data_request?id=lu_status2&output_format=json&DataVersion=932356278&Timeout=60&MinimumDelay=1500&_=1453931816609 tcp{client}: 0x216ed08
2016-01-27 16:10:53.360 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:10:55.381 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:10:58.919 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:02.963 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:07.006 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:11.049 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:14.587 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:18.631 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:22.674 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:26.717 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:30.761 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:34.804 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:38.342 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:42.385 luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:11:46.429 luup.io.incoming:: bytes received: 51, status: OK
– and it continues to receive the data from the Envisalink…[/code]

akbooer · January 27, 2016, 10:44pm

So is that better/further than before, or just different?

CudaNet · January 27, 2016, 10:49pm

Further.

When I clicked the ‘Read Panel’ button, the keypad chimed and the plugin status indicated busy. I never got this far before. It’s at this point that Cybrmage’s instructions indicate NOT to touch the keypad so I didn’t. I peeked at the logs and noticed errors. I waited a couple of minutes and it remained as-is. The link to the system remained persistent as the keypad didn’t cycle through all the faulted zones - rather just between battery lo and fire trouble.

akbooer · January 27, 2016, 10:54pm

[quote=“CudaNet, post:11, topic:190783”]Further.

Good!

The link to the system remained persistent as the keypad didn't cycle through all the faulted zones - rather just between battery lo and fire trouble.[/quote]

…so that would be there repeating messages, then:

2016-01-27 16:08:50.042   luup.io.incoming:: bytes received: 37, status: OK
2016-01-27 16:08:50.043   luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:08:54.085   luup.io.incoming:: bytes received: 37, status: OK
2016-01-27 16:08:54.086   luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:08:57.622   luup.io.incoming:: bytes received: 37, status: OK
2016-01-27 16:08:57.623   luup.io.incoming:: bytes received: 51, status: OK

There’s a nil reference error early on:

openLuup.scheduler:: job aborted : [string "[4] I_EVL3VistaAlarmPanel1.xml"]:8719: attempt to concatenate local 'isDisabled' (a nil value)

Gosh, that code has a lot of lines (8719 is the source line number)!!

CudaNet · January 27, 2016, 10:57pm

Yes x3. Cybrmage supports many Honeywell panels in this code so I’m sure that lends to it’s overall size.

[quote=“akbooer, post:12, topic:190783”][quote=“CudaNet, post:11, topic:190783”]Further.

Good!

The link to the system remained persistent as the keypad didn't cycle through all the faulted zones - rather just between battery lo and fire trouble.[/quote]

…so that would be there repeating messages, then:

2016-01-27 16:08:50.042   luup.io.incoming:: bytes received: 37, status: OK
2016-01-27 16:08:50.043   luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:08:54.085   luup.io.incoming:: bytes received: 37, status: OK
2016-01-27 16:08:54.086   luup.io.incoming:: bytes received: 51, status: OK
2016-01-27 16:08:57.622   luup.io.incoming:: bytes received: 37, status: OK
2016-01-27 16:08:57.623   luup.io.incoming:: bytes received: 51, status: OK

There’s a nil reference error early on:

openLuup.scheduler:: job aborted : [string "[4] I_EVL3VistaAlarmPanel1.xml"]:8719: attempt to concatenate local 'isDisabled' (a nil value)

Gosh, that code has a lot of lines (8719 is the source line number)!![/quote]

CudaNet · January 28, 2016, 3:48am

I just noticed I didn’t enable debug via the EVL3 interface (my bad). I re-ran the same test except this time I disabled AltUI’s debug and set my alarm panel to a default installer code (removes steps 8 & 9 performed earlier).
Here are those logs, much easier to read but nonetheless, the exact same result.

cybrmage · January 28, 2016, 4:39am

So… I retested the plugin with the new openluup files… and found a few minor bugs…

In io.lua, a new receive function has been added, but the read function still calls socket:receive(). This leads to the socket call blocking lua execution.
The read function calls socket:settimeout with the plugin provided timeout value (in seconds). This leasts to the read function having a millisecond timeout value (although the luasocket documentation specifies that the timeout parameter is specified in seconds, it also states that it has millisecond precision… and it actually uses the parameter as a millisecond value)
debug information for the luup.io.read call is not provided.

I edited the io.lua file to fix the bugs, and retested the plugin… and everything appears to work as expected (including panel and user code management functions).

I have attached the fixed io.lua file for openluup. I will post an updated L_EVL3VistaAlarmPanel1.lua file for the EVL3Vista plugin later tonight, as I amworking on removing the dependency on the nixio.bit library.

[quote=“akbooer, post:12, topic:190783”]There’s a nil reference error early on:

openLuup.scheduler:: job aborted : [string "[4] I_EVL3VistaAlarmPanel1.xml"]:8719: attempt to concatenate local 'isDisabled' (a nil value)[/quote]

That is due to openluup not supporting the devices “disabled” attribute… That error will make an ugly appearance for all of my plugins, as they honor the disabled attribute, and if the attribute evaluated to true they will exit gracefully.

It’s a simple fix…

Yep… 9062 lines in the lua file along… And it’s the medium sized plugin… Wink Connect has over 11000 lines of code and over 14000 lines of code for support file… it is a MONSTER!! 8-}

cybrmage · January 28, 2016, 6:00am

Here is the updated lua file for the EVL3Vista plugin.

cybrmage · January 28, 2016, 7:15am

I have also tested the Caseta Connect plugin (to test the “raw” protocol), and it works perfectly.

I have posted the updated plugin in the Caseta Connect plugin thread.

akbooer · January 28, 2016, 8:00am

What can I say? Feel free to pass a fine-toothed comb through all my code! Once again, my unit testing needs enhancement.

Outstanding work!

Thanks

CudaNet · January 28, 2016, 2:20pm

Thank you both for working through all this… The install went perfectly (of course) ! I just need to fetch the requisite icons and I can move forward with this project…

CudaNet · January 28, 2016, 3:07pm

Question

This particular icon file is present but its the :3480 that’s throwing me off… Looking at the other files being loaded, it’s fetching them from ‘{ip_address}/skins/default/icons/filename.png’ and that’s what’s throwing me off.
Am I overlooking something ? Here’s the error from the console and I attached a screen shot. All other icons are rendering correctly.

GET http://10.0.4.122:3480/skins/default/icons/vista_partition_device_keypad.png 404 (Not Found)