For a while now Im developing my own scripting for home automation with the Vera Lite and several zwave devices. Ever since Im wondering about what is actually wise.
In my opinion, when you build your own custom scripting using the API, you will always be forced to “poll” the api to get some data. For example: if you want the temperature out of a sensor, you make an http request to get the temperature. If you want the strength of light, poll the light sensor. And with such sensors, its enough in most of the cases, to poll every 60 seconds.
With a motion sensor however, you need a quicker response. When you enter a room and want the lights to be switched on, it needs to respond within a second. This means that the custom scripting needs to poll the API every 1 second to get behaviour.
In the GUI however, you can make the light sensor trigger a zwave switch. I think internally there is some kind of thread listening and waiting on something instead of polling.
This makes me wonder. Am i right with the above? Do I understand the differences correctly? And does the GUI do a better handling of things like a motion sensor?
Polling simply can’t get the kind of instant-response behaviour that you get with a Z-Wave motion sensor. For one thing, it would run down the battery on sensors in no time. So there is clearly some kind of subscribe/notify mechanism in place, where Vera has instructed sensors to report when their state changes, and Vera runs a listening thread to catch these notifications. For Z-Wave in particular, this is built into the spec. Z-Wave, being a mesh network with nodes acting as repeaters, makes it practically a given that if a sensor trips, Vera is going to hear about it. Vera still polls, as frequently as every few minutes or hours, but this is primarily to correct any state changes that weren’t detected, or to collect slow-changing values like battery level. All of Vera’s Z-Wave interface is hidden inside he LuaUPnP engine, so you can’t actually use any of this knowledge in plugins.
For plugin devices like the ones you and I write Lua code for, whether you are stuck with polling or can use instant notification depends on the capabilities of the device you are taking to. Some, like the Philips Hue, simply have no capacity to inform anything upstream of a status change. For those, you poll. Some, like many alarm systems, have a custom serial protocol which developers can listen to and get immediate notification. Some devices use a standard protocol such as UPnP, which requires plugins to request a callback should the device status change. Requesting a UPnP callback (“subscribing”) takes quite a bit of coding, and listening for the notification is a bit hackish in Luup because of limitations in Vera’s web server implementation.
To summarize: every protocol has different capabilities and occupies a different place on the poll/subscribe spectrum.
… And to address the other part of your question: you won’t get very far with a plugin that polls once a second. The way that plugins get notifications on demand is to define actions which the remote device will call when it needs to. Essentially, your plugin coaxes the device into sending an HTTP or serial message, in an acceptable format, when the device’s status changes. The Luup engine listens on both of these interfaces, serial and HTTP, on an internal thread, so your plugin doesn’t explicitly contain the listening code. The Luup engine then invokes an action on your plugin and your plugin acts on the information. Bingo, instant notification.
I think I elaborate a bit more on my setup since I find it difficult to map your explanation to my personal situation.
As said before, I have a Vera Lite. Several devices; some switches, and a 4-1 AEON sensor with motion detection.
Currently I build my logic in my own scripts instead of the GUI. My scripting is JavaScript which I run serverside on a separate linux box, using node. To get information or to perform action I use the HTTP api that is available on the Vera. When I say polling, I think Im not actually polling, but only performing an HTTP request to the Vera which gives me the latest available status of every device connected to the Vera(I don’t expect any battery drains because of this setup).
Having that said; I understand the bigger part of your post. Only the second (part of the) post is raising some questions. Focussing on a solution for my situation Im wondering if I can make it work in a proper way with JavaScript or any other scripting that is using the HTTP API. Personally I don’t think so but you conclude your second past by stating “The Luup engine then invokes an action on your plugin and your plugin acts on the information. Bingo, instant notification”.
Do I understand correctly that such a solution only applies when using LUA code and building a plugin? Or can I achieve such a thing also by using the HTTP API and custom scripting instead of using LUA for making a plugin.
Hopefully I made my situation and questions clear.
Aaah, that was important information which you withheld from me until now You can probably ignore the stuff I’ve said about the Luup engine and plugins; that’s only relevant if you are attempting to interface to a custom device directly on the Vera.
You are going to end up writing a remote-access client, like the MCV UI Simple example. That wiki page talks about the “long poll” which you would need to invoke on your Linux box. It’s reasonably efficient in that the HTTP request will block until something on the Vera changes, or the request times out. Your script will have to look at the result of that long poll, and decide if it needs to act on it before doing the next poll. Repeat forever.
If you want to avoid even the long poll, then you would need to add code directly onto the Vera, have it watch (luup.variable_watch()) the motion sensor for changes (or create a scene in the Vera UI to watch it), and have Vera run a bit of Lua to push the status change over to your Linux box with (say) a custom HTTP request that you’ve taught your Linux box to listen to. This kind of thing has been discussed on the forum before, usually in the context of sending Prowl notifications to your mobile when a motion sensor trips, or of invoking a command on the Linux box with an SSH command to start your security-camera-recording software.
Okay, sorry for the lack of info in the first place.
Thats not a “walk in the park” thing to setup I guess. I do like a challenge though. Thanks for pointing me in the right direction. I will figure out if this is the best possible way for me based on the documentation/topics I can find.
If you want to avoid even the long poll, then you would need to add code directly onto the Vera, have it watch (luup.variable_watch()) the motion sensor for changes (or create a scene in the Vera UI to watch it), and have Vera run a bit of Lua to push the status change over to your Linux box with (say) a custom HTTP request that you've taught your Linux box to listen to.
Okay, I have been thinking about this for a while. And this option feels about right. You mention to add the code directly to the Vera. Where should I add that specifically? I’m totally new to lua/luup, so this is gonna cost me some time anyway but for now Im not even sure where to add it and run it from.
After the luup.variable_watch detects a change, it can execute a function. A function that does a http request to my linux box. Do I understand correctly that I need my linux box to run some kind of webserver(in my case with a piece of javascript run by node) that performs some action/function based on the request done by the Vera?
And one other thing Im wondering about, is how the luup.variable_watch works under the hood. The main reason Im looking at this alternative is that it doesn’t feel right to spam the http api on the vera every second to check the status of a device(for example the motion sensor). But if the watch function does more or less the same under the hood, Im not sure if its still worth to go all the trouble.
In the “end state” of my home automation I expect maybe 5-10 motion sensors, and 10 door sensors. It feels like the http api on the vera should be able to manage those 25-50 transactions per second… but that is just a guess.
You’d add it to the Vera’s “Startup Lua Code”, which you can access from Apps > Develop Apps. Here is an example which watches a variable and performs an HTTP request as a result.
Do I understand correctly that I need my linux box to run some kind of webserver that performs some action/function based on the request done by the Vera?
Some kind of server, yes. I mentioned web server because it’s easy to set one up on a Unix machine and it’s easy for Vera to make an HTTP request. But you could equally do it over SSH or some other protocol. There are examples here on the forum for having Vera run SSH commands remotely on other computers too.
And one other thing Im wondering about, is how the luup.variable_watch works under the hood. The main reason Im looking at this alternative is that it doesn't feel right to spam the http api on the vera every second to check the status of a device(for example the motion sensor). But if the watch function does more or less the same under the hood, Im not sure if its still worth to go all the trouble.
Beware the sin of premature optimization! Until you’ve tried the simple solution and found it to be inadequate, don’t eliminate it from contention. I think that you’ll find for your sized system, the long poll will be perfectly adequate. It’s no more of a load than the Vera web UI; I know that because the Vera web UI is based on just the same long poll.
It’s never been revealed how luup.variable_watch() works underneath. That’s behind the API so you shouldn’t even be thinking about it. But as a data point, I have dozens of them running on my Vera.
Some kind of server, yes. I mentioned web server because it's easy to set one up on a Unix machine and it's easy for Vera to make an HTTP request. But you could equally do it over SSH or some other protocol. There are examples here on the forum for having Vera run SSH commands remotely on other computers too.
So in my case I would just make the vera box initiate a specific script by starting it over ssh on the commandline, right? For examle my js script starting with nodeJS.
Beware the sin of premature optimization! Until you've tried the simple solution and found it to be inadequate, don't eliminate it from contention. I think that you'll find for your sized system, the long poll will be perfectly adequate. It's no more of a load than the Vera web UI; I know that because the Vera web UI is based on just the same long poll.
So if I’m correct, you’d first try the long polling option before trying the luup.variable_watch way? Im not really familiar to both, but partly the reason why I picked the variable_watch option is because it felt more straight forward to me. All the long polling examples I’ve found on the internet felt complex to me. The variable_watch solution however, especially with the ssh solution, seems more obvious. Even more with your explaination. “Just” add the watch thing, have it execute commands over ssh, and Im settled.
With the long polling I need to start some script on my linux box with node. That script is acting as some http “server”, listening and keeps listing until a certain request made from the Vera. Thats the part I think I might be able to setup. But then still I need something thats actually making a request from the Vera, based on “something”. I feel I still need something like the variable_watch here.
Maybe I’m completely off track and overlooking crucial facts… but can follow me?
No, the long-poll method is totally a client; the server is the Vera HTTP server in this case. You are confusing the two methods I described. Go back and read my original reply. Everything up to “Repeat forever” is you writing a client. That’s the long poll. You don’t write any code on the Vera, so Vera is not making any “requests” to your script.
Edit: But if you want to go the luup.variable_watch() and SSH route, go for it. I just wouldn’t do it on my network, more for reasons of trust than of efficiency. But I digress.
I’ve been reading it over and over again trying to get it
From the things I read on the internet I drew the conclusion that the Long Polling mechanism sets up some kind of http webserver(piece of JScode) with some kind of condition in it. When a request is done to that server from the other side(in this case vera) and matches the conditions, it(webserver at linux box) can perform an action(for example switching of a light). But Im a bit confused about all of this right now. After your previous statement it feels like I’m overcomplicating things.
If I only need to do something on my linux box(in this case the client), how does it all work together. From what I understand is my linux box waiting on something to change. But where is it looking at? Can you give an example how this would work?
Sorry if I’m asking a lot. And if it seems I’m not looking into this on my own, I can assure I do google and read a lot. I just don’t get it yet.
In my situation, I do something like this(general example):
Yes, you are overcomplicating things (sorry). Also, some of the references you are reading are perhaps explaining how to set up a server when the server already exists in the form of Vera. You should only be writing client code.
So you’ve got a loop that periodically queries Vera for its state. Keep that. Now imagine that Vera, when it gets that request, instead of responding immediately, blocks its response until something (anything) changes state on the Vera. Your script is now effectively paused in its HTTP request. Then when Vera notices that a variable on a device has changed, it returns the response, and your code resumes.
With Vera forcing your script to sleep, you can remove your own sleep statement from the loop as redundant. Now, if nothing changes state on your Vera (this is very common), your script spends almost all of its time blocked and not using CPU cycles. If something does change then your script finds out about it instantly. (It may not be a device that you care about, but that’s for your code to decide.)
That’s what the linked UI Simple pseudocode is doing.
Also, some of the references you are reading are perhaps explaining how to set up a server when the server already exists in the form of Vera. You should only be writing client code.
I think you are exactly right here. But I suspected I had something to do on the Vera aswell indeed. What you are saying is that the Vera is capable of blocking the response? If you can elaborate there, that would be of great help.
I can remove the sleep from my loop, I understand that part. But I do need to KEEP the loop because I want to keep my script monitoring not only the first change, but also the next changes, is that right? In JavaScript I think I still need some kind of interval setting, but that wont harm me since the Vera keeps blocking the request and after a change, then it will return the response. My script continue’s and starts the next loop(with or without a short sleep/interval).
Am I with you so far… well I have the feeling I am, but you can judge better may after reading the above. The obvious next question for me now is how the Vera blocks the response. How to achieve that mechanism working.
Thanks for your time investing in my learning curve futzle. Its quite steep for me at the moment but you are a great help.
Yep, you’ve got it. That “holding back the response until something changes”, that’s the long poll mechanism.
Enabling this requires no work on the Vera. You just have to supply the right options in your HTTP requests. I forget what they are exactly but you can find them on the UI Simple page. There are a couple of Unix-style timestamps and maybe a timeout duration too.
Edit: loadtime and dataversion are the timestamps. The timeout is, unsurprisingly, called timeout. Also there’s minimumdelay which you can use to prevent overloading Vera or your script.
You can probably ignore the stuff I’ve said about the Luup engine and plugins; that’s only relevant if you are attempting to interface to a custom device directly on the Vera.