openLuup & (not) vera

HSD99 · March 10, 2020, 9:25pm

May I strongly suggest that you document everything you did while it’s fresh in your mind?

rafale77 · March 10, 2020, 9:28pm

Aye aye captain! I will start working on the readme of the zway bridge as early as tomorrow. I had quite a few exchanges with the CTO of zwave.me who helped to get here so… not everything is intuitive.

openluup has also gone through a few update iterations.

HSD99 · March 10, 2020, 9:30pm

You know how it is…decisions that are totally logical and clear today will look like gibberish when you revisit them in six months!

DesT · March 10, 2020, 11:00pm

It’s done!
Running zwave at 100% from zway bridge on openLuup…
The only thing left on vera it’s Zigbee for some thermostat and the NEST plugin until I completely change all my NEST Product (thermostat and protect) for something zwave too!

thanks @akbooer for the most part, and thanks @rafale77 for your help too!

rafale77 · March 11, 2020, 12:02am

I just found a crazy problem…
The Zway commands are so fast that for my ecovents. it sends the get position so quickly after the set position that vent has not yet had time to move and responds with its initial position or some position in between while it is still moving… It’s pretty funny but a problem to solve. It reminds me of the countdown timer plugin which was running so fast on openluup that it did not have time to see the variable trip at the end of the timer and would therefore never trigger anything.

DesT · March 11, 2020, 12:12am

@rafale77 I think it’s a good problem, being so fast now that I’m enjoying having almost a realtime response

rafale77 · March 11, 2020, 12:21am

yeah well… it means I may have to resend a “get command” after some delay and most likely will do it from the bridge.

rafale77 · March 11, 2020, 4:12am

So to do this, I think I am going to want to create a binding to the function “poll node” from Vera to Zway. I will get on it when I get some time…

Alright, I think I got it. Sending pull request… @akbooer. Polling nodes now works from openluup.

akbooer · March 11, 2020, 10:23am

This happens for my RGBW LED strips, and is one reason why I don’t use the feature to change dimmer levels when switched off. However, for the lights (Fibaro driver) AltUI eventually catches up with the final level. It’s also the reason that the on/off switches twitch sometimes when switched on or off.

rafale77 · March 11, 2020, 1:40pm

This interesting. I am getting in touch with their staff to see how this can be resolved. We can patch it by adding a poll after a few seconds which I validated to work but I am not sure I like the additional traffic it generates. It may be better to have them implement a delay between the set and the get.

rafale77 · March 11, 2020, 11:23pm

Alright I got started.

github.com

rafale77/Z-Way/blob/master/README.md

# Z-Way for OpenLuup		
 Zway introduction as a replacement for vera		

  Why Migrate? If you are reading this, you probably already know but just in case...		

  Because Z-way is a far more stable platform than the vera especially on larger networks. The vera current state has such fundamental design flaws that it is nearly unuseable for any network above ~40 devices. It becomes too chatty and error prone.		
 Even on smaller networks, the notoriously absurd abuse of luup reloads and forced deletion and creation of devices has turned away more than one customer.		
 A quick visit on the vera forum will get you a flavor of the issues plaguing this platform ranging from device bricking, poor storage management using tiny portion of what the hardware provides, ghost sensor trips, memory corruption, memory leaks... All of which I have experienced.		

  Z-way-server on the other hand is built on a much more stable and solid API and zwave library. The "Expert UI" can pretty much support any device which has nothing propriatory. It's smarthome UI, which API this plugin relies on however has its flaws, filtering out useful command classes and requiring some workaround to address the lower level API.		

  Assuming that the entire automation setup (scenes and plugins) have already been migrated to openLuup, you can now migrate the zwave hub and take advantage of the best part of the vera, it's object data structure, API and lua/plugins, and get rid of its instability.
 
![alt text](https://anhman.neocities.org/Hub.png)
  
  Having tested a wide number alternatives, including some roadmapped ones, I concluded that, starting from a vera setup, this solution is the best path to a stable and scalable system. The takeover of mios by eZLO now focused on a new platform has also not yielded much hope for a migration path or even for a viable solution in terms of performance, reliability and security.
  
  
  # Guide to migrate from Vera as a zwave device hub to Z-way

This file has been truncated. show original

HSD99 · March 12, 2020, 12:06am

Bravo! Excellent work and thank you for sharing!

slelieveld · March 12, 2020, 6:13am

Bookmarked the page and waiting for more pictures and layout and others before to start

Catman · March 12, 2020, 7:19am

DITTO!

Thanks so much

C

rafale77 · March 12, 2020, 4:05pm

Alright, that is motivation! I will continue working on it and will try to insert pics and diagram so it doesn’t look so much like a book.

Noticed that the title of this thread changed… to a very strange one.

I am on day 3 of the conversion and had spent quite a bit of time preparing it prior to doing it. I am still working through a few kinks here and there for more complex devices. Hopefully this will pave the way for others to smooth things out. A lot of work has gone into the zway2 plugin and to set the expectations right, @akbooer is now working on refactoring it. It was a “prototype” long abandoned which he revived over the last couple of weeks. It could use some restructuring and clean up but works amazingly well for its age and purpose.

The binary switches and dimmers are fairly straightforward. For the dimmers I added a poll routine in my startup lua if the target level is different than the read back after 5s to make up for z-way being too fast at reading back the device if the device does not support instant status.
The security sensors in general went pretty smooth. A lot of work has gone into the multi-sensors and they work ok now. I am seeing only a problem with these multi sensors not being able to be armed by a service action and therefore the UI. I am arming them directly using a variable set.
I also replaced the housemode sensor arming with a couple of simple functions in my startup lua and are called by scenes triggered by housemode changes.
Various scene controllers like the leviton 4 button scene/zone controllers and minimotes are not officially supported by zway but work fine through the Expert UI configurations and associations. As mentioned above I even integrated a function in the bridge to control the LED which to me is better than what the vera uses.
Thermostats being complex devices with a lot of command classes to cover is an area which has degraded: The essential functions work naturally: Setting the temperature targets for heat and cool, as well as temperature readback is transparent. The issues have been around setting the cool/heat/auto modes which Z-way’s smart home UI is very weak on and does not report properly. So I addressed it by sending commands to the lower level ZWaveAPI and not reading back from it. Likewise for the fan modes. The thermostat and fan state to this point cannot be read back. Z-way’s expert UI reports it correctly but the plugin reads back from the higher level SmartHomeUI to which these command classes are not reporting. As a result, openluup doesn’t report the instant state of heating/cooling or circulating air of the thermostat which it would do through the vera.

[Another thing is the management of “dead” or unreachable nodes done on Z-way is also only down at the ZWaveAPI level and therefore is not reported back to openLuup. It means that I have to go to the Z-way interface to check on “cannot detect devices” unlike what I had using the vera where I had it send me push notifications when a node was out of service.]

Edit: Scratch that section above. Turns out the new version of Z-way actually implements a variable reporting failure status which has now been implemented in the bridge if @akbooer pulls my new edits.

This is about it. Will report if I find out more but so far it is looking very good.

rafale77 · March 12, 2020, 6:01pm

Another post on the effect on the vera. (It is still holding my zigbee devices which will eventually migrate to HA)
As @therealdb first reported the vera suffers from a memory leak when the ReloadOnTimeJump option, provided since 7.30, is disabled. I chose to disable it because my assessment of the Luup reload for these types of situation is worse than doing nothing. The downside was that it disabled all time based scene executions which I did not care for since my scenes have been on openLuup. The leak gets reset by a luup reload.

Well from the graph below is the memory profile over the past 7 days. After disabling the zwave radio of the vera, the memory leak appears to have stopped. Well a few more days might be needed… It may be that the leak is now much smaller since I went from 232 devices (including children) on the UI to 29.

slelieveld · March 12, 2020, 6:26pm

The title of this topic was change multiple times by vera… It is a very active topic and I wonder why they change their own title… again. It was “Vera alternative” and before something else

Also hope that by seeing that people are looking for alternatives it is a clear signal that their current product is THE major product of their existence. The other one, Ezlo, is not very active yet.

Thus, having and keeping their users happy is the most important thing.

And that means keeping the Zwave users happy! Cause after all, vera, zwave is what it all started with and what you are known for. Focus on that please!

therealdb · March 12, 2020, 6:29pm

Their choice is to start from scratch and I respect their courage, but I’m not sure it’s the right path for me, personally. As already said, the new firmware seems to have less features than the current one, and the latest fw iterations, with a lot of regressions, seems to encourage their view that the current generation is fundamentally flawed. We’ll see, this topic remains very interesting, even if it’s probably too difficult for me at the moment.

rafale77 · March 12, 2020, 6:56pm

The development strategy on the new platform is disturbing to say the least and I have not been shy about it. Yes building a new platform and forgo the old unstable junk is courageous and needed but…

The platform is built backwards. The fact that they are working on the zwave library now, one year later is a recipe for disaster. It’s not a building block, it’s a cornerstone of the foundation. So many things about the platform will need to rely on this library. The command class support looks good but only now?? They say it will be shared across all products, great but what was the alpha linux built on if this was not ready?

The architecture should have been in this order because they all have dependencies on the previous layer:
The protocol serial API (provided by zwave)
over which you build the zwave translation library
on top of which the API is built
And then the logic/application layer
Then at last the user interface followed by the mobile app
And then an optional cloud assisted layer.

The fact that they have no migration strategy whatsoever and that the new API is fully incompatible with the previous one, That the very basic structures for plugins development appear to be secondary priority to I don’t know what yet, just forced my hand.

Overall impression is that the direction is: “Lets wipe everything clean and get rid of everything which worked first. Let’s keep what was broken and work on it later.” The fact that they cannot fix the current platform, that the focus on the new platform has been on changing things which did not need to be changed urgently, gives me no confidence that the new one will be built better… in spite of the better architecture…

akbooer · March 12, 2020, 7:02pm

I would say that openLuup+ZWay is an existence proof that the old platform could be fixed.