openLuup: Asynchronous I/O

Like this…

for i = 1,100 do
  local response_table = {}
  local ok, err =  async.request (
    {sink = ltn12.sink.table (response_table), ...},    -- plus other request stuff 
    function (response, code, headers, statusline)
    -- do whatever you need with the response here
    end)
end

Sure, but suffice it to say, this is going to focus people’s attention on scope. There’s a lot of lazy scoping out there, and that won’t pass muster in this context.

caveat implementor !

2 Likes

Hello ak

Been having a read of all this. Have we got any access to async sockets amongst all this of late? Or are we still stuck with having to use luup.io.intercept()? I’m thinking both Vera & openLuup. I wrote a Paradox alarm web page scraper years ago and it works fine. The plugin works using polling and logs into the alarm and then extracts the sensor status from the web page it reports.

I would really like to just use some async socket and get the alarm sensor data via a callback, as soon as it occurs, rather than after some fairly long poll delay, like I have know.

Any ideas on this one?

Hi,

The Vera build in is still the luup.io indeed. The way i worked around it for the Harmony plugin is to schedule a task each second and use socket:select to see if there is any data. Yes, process it. No do nothing. This is working nicely on openLuup and Vera. You can look at the Harmony plugin source for details : vera-Harmony-Hub/L_Harmony.lua at master · reneboer/vera-Harmony-Hub · GitHub
look at the wsAPI bit starting at line 818

Cheers Rene

The asynch_http module essential works the same way, using socket.select() as, indeed, does the openLuup scheduler.

So,in fact, you’ve always been able to use async sockets.

I wrote a proxy that runs directly on the Vera, or on openLuup, to provide async sockets with minimal changes on both platforms. It allows you to structure the plugin so it works both with and without the proxy, so if a user doesn’t have it installed, you plugin continues to work the “old” way (if you let it), but will be much more responsive if you use the notifications. Every one of my plugins that uses TCP sockets for communication now uses it, with and without SSL.

SockProxy

I also wrote a full WebSocket implementation (except 64-bit length frames, which I’ve not yet seen used in practice) on top of it, which again, also works without it.

3 Likes

@rigpapa Hey Patrick, I’m taking a crack at this with the EVL plugin. I’m not good enough with Linux to understand the pros and cons of using a systemd service file in place of the init.d file you suggest (I’m using Ubuntu). Are there any gothchas with the plugin if I just use the “before:” Systemd action to start things prior to openLuup.

I think any way you get it started is fine.

OK Thx. The proxy sits between the data emitting server and the data receptor (openLuup in this case). Is the sockproxy config file used to specify the listening parameters for both servers? I’m not clear on whether the proxy is listening for a config-described vera connection, or a config-described connection from the server supplying the data, or both.

That depends on how you want to use it. The README file describes the possibilities, so make sure you study that.

Most of the time, I just connect to the proxy running on the Vera or local openLuup system (via 127.0.0.1 on port 2504). You do this instead of connecting directly to the API you are going to talk to. You then ask the proxy to set up the connection by issuing a CONN command over the connection to the proxy. The proxy then connects to the remote host, and goes into “passthrough” mode, where you cannot send any further commands to the proxy, but rather any data you send to the proxy goes to the remote, and any data you try to receive will be that which is received from the remote. The proxy will, however, be sending you notification actions when data is ready for you to receive (so you don’t have to poll/wait, you just read the proxy socket when you get a notification action). You do not need to do anything in the config file for this approach.

The other way you can do it is to have the proxy pre-connect to your remote and set up a tunnel in advance. This is done through the config file: you pick a local port and tell the proxy that when it gets a connection (from you) on that port, it should automationcally connect to the remote on the IP/address and port your designate. I regard this as a slightly more fragile setup because it requires, in order to receive notification actions, that you put the device number of your plugin in the config file–that could change, and if it does, it requires you to change the configuration file and restart the proxy. This is why I prefer the CONN method.

Most of my plugins that need async communication now are “proxy aware” and operate in the zero-configuration approach. They first try to connect to the proxy (127.0.0.1 port 2504), and if successful, issue a CONN command for the remote and then everything else is business as usual. If it can’t connect the proxy, it connects directly to the remote and goes on with business as usual (although in this case, you must poll for receive data as the proxy is not involved and therefore not sending notifications).

Hey Patrick.
So this plugin is way cool… I have the basics working and will soon convert a couple of obsolete plugins that I’ve hacked to death anyway… However, I do have a couple of requests:

The readme.md under openLuup install should mention that you install the sockproxyd luup device in the standard openLuup way (which is slightly different from a Vera install). I originally assumed that, for openLuup, the luup device would be created when running the startup lua file.

The lua file for the luup device itself has the basic startup linux commands embedded in the lua code. Could you condition running this code on whether one is in a Vera environment or an openLuup environment as I’m using systemd to start the proxy. Here is my service file if anyone wants to comment or improve “sockproxy.service” :slight_smile:

[Unit]
Description=SockProxy for Vera plugins
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStartPre=/bin/sleep 10
WorkingDirectory=/etc/cmh-ludl
ExecStart=lua5.1 /etc/cmh-ludl/sockproxyd.lua -L /etc/cmh-ludl/veralogs/sockproxy/sockproxyd.log
Restart=on-failure
RestartSec=10
StandardOutput=syslog
StandardError=syslog

[Install]
WantedBy=multi-user.target

The service file needs to be placed in /etc/systemd/system (set the appropriate properties for the file) then type at the command prompt “systemctl daemon-reload”, then “systemctl start sockproxy.service”. Finally, when you have things working, “systemctl enable sockproxy.service” which autostarts the proxy at boot.

This approach avoids the need to run a daemon and its attendant .sh scripts to start and stop the program. (systemctl start/stop sockproxy.service). This is why I was asking to condition the linux commands to vera vs openLuup environment as I imagine openwrt requires the linux approach you outline in the documentation.

If a plugin uses luup.io for reads and writes, do I need to change these commands to something else to make full use of the proxy?

Finally, (for now anyway!) in the device itself, in the control UI, could you place a hyperlink to the sockproxyd.log. I imagine you would create a variable to hold the log path that could be modified by the user, and then connect the hyperlink to the file path. This would be helpful in diagnosing connection issues.

Thx for this, and as I said, very cool.