I have been working on slowly making my vera plus just a bridge due to its hardware limitations, disproportionately overambitious integration and poor core software development.
As the next step in this journey, now that I have absolutely no plugin integration requiring internet access, I have decided to block all internet traffic to and from the vera unit and see how it fares and wanted to share my experience and maybe get inputs from people who have already done so:
First thing to do is to disable the auto-update from the apps which are left. For me it was easy as I only have ALTUI, System Monitor and one instance of the countdown timer left.
Ohh I forgot those 2 absurdly useless plugins from vera: The Sercomm IP Camera and the Amazon Alexa Helper. I tried deleting them thinking they could not be re-downloaded after my vera has no internet access. Well I was right but the vera after about a minute post luup reload tries to get them again, post an “error in scene or plugin” in the luup task bar (The blue message bar) and I discover that it disables its network socket. Basically all scenes continue to work unless it is required to send anything over http. The log shows that there is an AutoPluginInstall function which failed. I decided to let it redownload them and now I am seeing This plugin is automatically available on all Vera controllers and does not need to be manually enabled. on the “my apps page” for these two plugins. I can’t say I am pleased. MCV: Your hardware is not capable of handling all the integrations you are throwing at it. Please allow your customers not to download software they do not use. Or if you want to do so, upgrade your hardware by 2-3 generations on memory and CPU. Heck, my coffee machine has more processing power and memory than the vera.
The vera seems to be… less prone to random luup reloads. I am running the bet 26b firmware.
My access through VPN and Homewave to the vera is now much faster through VPN having bypassed the mcv cloud server.
No more “your vera is offline-online” nonsense. It is permanently offline
I setup all my notifications through Homeassistant and Pushover which is much more reliable and has less latency.
I think the system is more secure now having cut off one uncontrolled access path to my home.
Great reliability improvements overall as I won’t have unwanted downloads onto my Vera updating files sending data etc.
I lose the daily automatic backup and log transfer… So I have to backup locally and log onto a USB stick.
As many people already know here, keeping time between reloads is a big issue for the vera. I did set it to read time from my local NTP server so it doesn’t need to access a public NTP. What puzzles me is, the vera updates the user json file regularly. It is how it is able to remember status of devices between reloads. Why not get the time from the last modified date of the user json file by default before going for the NTP server??? At least it won’t be off by 19 years.
Overall pretty happy with the outcome and I may keep it longer than I initially thought if it stays this way.
Last year I had to deal with sporadic internet access and frequent reboots due to construction and change in my electrical setup. I opened a ticket and they told me that persistent time was in their radar. One year later, no joy.
So even with the two rogue apps installed, I am still occasionally seeing Luup reloads failing to complete. It seems to be timing out. In which case I have to disable my firewall blocking rules just for it to complete reload and then re-enable. I am suspecting it to be due to ALTUI. Will see with more testing. That being said, I have not seen a single sponteneous unwanted luup reload which is encouraging.
One further update: I had a scene mistakenly run this morning because of a failing
return luup.is_night() returning true when it is supposed to be false.
When I go run this in the test Lua code, it returns correctly and the time on the vera is correct.
This is going to push me to move my time related scenes from the vera to openLuup.
I am 100% convinced now that the reliability issues of the vera are related to the UI7 trying to do too many things on a very limited CPU. I believe that the scene failed because it Luup.is.night code did not return fast enough for the scene to continue correctly. I am still quite frequently seeing the UI missing zwave sensors untrip messages. It happens on all kinds of sensors, not a single type. It is also very inconsistent: The same sensor untrips fine for days and just once out of the blue would remain tripped until I go back and trip the sensor and untrip it but walking in front of it or opening and closing a door. It ultimately is problem of jobs and threads running out of sync with others and a fundamental architectural design flaw of the UI and gets worse as the system grows and more functionalities and integrations are added. I do not believe that there is an issue with the zwave or zigbee radio or network since the inconsistencies occurs even when they are not involved. A number of complex plugins which had occasional reliability issues, run perfectly once moved to openLuup. I also am convinced the many random Luup Reloads are triggered due to accumulation of errors related to job completion timing issues.
One interesting discovery… When the vera runs a luup reload without internet, it actually skips the loading of a few things. One of them is its lua socket so it is essentially not capable of reaching other ethernet connected devices through lua code. So I am currently having to open access to the vera, run a luup reload and then close access again. This is a bit absurd, not having internet access or access to the vera server does not mean we don’t want the vera to communicate on a network…
rafale77, thanks for this thread. I’ve been moving in this direction myself, albeit much more slowly. I haven’t done the detailed analysis of Vera’s interactions with the outside world that you have, but have certainly seen that the system relies on outside connectivity to a much greater extent that I had first believed. Here’s what I’ve done to date:
NTP redirected to local server on my LAN
The issue of system time is frustrating. Local NTP insures that Vera will be able to get system time when required, but, as you point out, using the timestamp on the most recent checkpoint makes more sense than resetting the time to January 1, 2000. Loss of time across programed events (Luup restarts) is simply poor system architecture.
All access to Vera from outside is done via VPN (no use of Vera’s relay servers)
Cloud backup-I’m OK with this, but wish there was an option to backup to a local network drive instead of, or in addition to, the Vera cloud.
I created a cron job to run a bash script daily and create a local backup. Just replace the necessary input with yours and then edit your cron jobs. I am doing this on the VM which runs openLuup but it can be done on any computer on your network.
Here is the bash script:
#!/bin/bash
backup_location=#path to your local backup folder#
max_number_of_backups=30 #I want to keep a month's worth of backup#
# Setup environment
. /etc/profile >/dev/null 2>&1
# Go to Vera backups directory
cd $backup_location
# Create a date-specific backup name
export BACKUP_NAME=vera_`date +%Y%m%d`.tgz
# Backup ZWave dongle to Vera (this is async and takes time to complete)
wget -O /dev/null "http://#veraip#:3480/data_request?id=action&DeviceNum=1&serviceId=urn:micasaverde-com:serviceId:ZWaveNetwork1&action=BackupDongle&Restore="
# Wait for Zwave backup to complete (actual time depends on # of zwave devices -- do the timing and adjust)
sleep 60
# Backup Vera (whcih will now include ZWave backup)
/usr/bin/wget -O "${BACKUP_NAME}" http://#veraip#/cgi-bin/cmh/backup.sh?external=1
function number_of_backups() {
echo `ls -1 $backup_location | wc -l`
}
function oldest_backup() {
echo -n `ls -1 $backup_location | head -1`
}
while [ $(number_of_backups) -gt $max_number_of_backups ]
do
rm -rf "$backup_location/$(oldest_backup)"
done
exit 0
Edit:
To complete the instructions in case people are less familiar with linux:
Create a text file above and name it “verabackup.sh” or whathever you want to call it. Remember to edit the ip address of the vera, the number of files you want to keep and the path to the folder where you want the backups to be saved.
Copy that file to a folder you like just remember it’s path on the server you want your backups to be.
Make this sh file executable by sending the command: chmod a+x verabackup.sh from the folder where this script file is
Edit your crontab by executing crontab -e and at the very end of the file add 0 4 * * * /path_to_script/backupvera.sh Don’t forget to replace the path and the name of your script file. This is for the script to run at 4am everyday. You can change this to whenever you want. The first 0 is the minutes and the 4 is the hour. Do not forget the asterisks.
Figuring out how to do a network backup was a future project for a Saturday with absolutely nothing else to do. I’m on the road next week, but will take a crack at this when I return.
Post 1000 and I hope to make this one useful and helpful:
I created a schematic of how my entire home automation system works and decided to share it.
After months of tinkering I have settled with a 3 controller system which is 100% local.
My graph shows all the Cloud APIs and connections needing an internet connection in blue and all the Local APIs in green with local communications in brown.
My main controller is openLuup running on a VM on my NAS. It shows all of my devices and runs most of the automation.
I have now successfully fire-walled the vera completely from the internet. I am still seeing it as the most unreliable of the 3 controllers and have dumbed it down quite a bit and will still continue as much as I can. It is essentially now a glorified zigbee and zwave radio with minimal amount of scenes I have kept there to reduce latency for scenes which are triggered by zigbee and zwave devices.
Home Assistant runs a number of cloud integration devices which are not available on vera/openLuup running on another VM. It is a secondary controller acting essentially as a bridge for openLuup. I am fairly new to it and I know I could consolidate a few more things within this controller.
Conceptually the vera is now like a Smartthing except that I have replaced the unreliable ST cloud processing with my NAS running a combination of openLuup and Home Assistant. In my view, with UI7 the Vera controllers really aren’t capable of being more than a radio: As you can read through a number of thread in this section of the forum. The hardware is just much too limited between the slow CPU and lack of memory/storage causing a number of issues as setups grow larger and I am continue to discover has fundamental reliability flaws. The major advantages of this architecture is, resilience to internet outage, as it would just cutoff inputs a few sensors and actions of even fewer ones, and reliability. I still get occasional mysterious luup reloads on the vera but it is much less frequent and less critical.
For this milestone post I wanted to especially thank all the folks on this forum who contributed to this setup and from whom I have learned: @amg0 for ALTUI, ALTHue and Iphone Locator @Akbooer for openLuup @Reneboer for Harmony @watou for Ecobee @lolodomo for Sonos @futzle for Countdown Timer @blakem for VRainsensor @airedale for Plantlink @Rigpapa for SiteSensor @intventlr for Homewave @bwssytems for the ha bridge @hackworth for the Homekitbridge
I may be headed in this general direction as well. I’ve just gotten into HomeAssistant and it’s another learning curve, but my RPi house controller (I have other processes running on it for over a year now) has been rock solid. How does HomeAssistant handle Vera doing a Luup restart? Does it catch the actions up afterwards or are they lost?
Having played with Home assistant now for a while, I don’t think it will have as frequent polling rate as openLuup. This means that a Luup reload will not affect it much. If it happens while a command is sent, the vera may still execute it when it comes back up. This is the reason why I am still maintaining openLuup as my main controller: I am more comfortable with lua coding and the interface. Home Assistant’s GUI while very configurable is still not as functional as ALTUI in my opinion. The lower poll frequency also means that you will have more latencies between the controllers.
For the past couple of days, I have been fighting with the vera having random luup reloads, losing time in spite of having set the ntp servers to local and losing its lua http socket. I finally got to the bottom of it this morning. It appears that I had set up my firewall a little too aggressively and blocking vera from some of the local network communications. I think I have now resolved most of the problems. What I am left is still random luup reloads, and intermittent periods of long delays in zwave commands and even occasional complete miss of a signal or command. Upon full vera reboot, I sometimes observe that the vera takes a long time before it get’s its ntp time and therefore goes off executing unwanted things. It’s not a big deal if you expect it and do not reboot the vera so often which I don’t, but it is something to be aware of. It’s not quite there yet but this is the right direction to get the vera more reliable. My overlay partition is stable at 54-55% used space with now only 2 plugins: Countdown timer and System Monitor.
Back to Home Assistant, I am considering using the sonos component with the embedded TTS function to replace the http API I am currently running for all the announcements and the roomba component to replace my separate API. This would consolidate a little.
I like openLuup as well but never had much luck running it on an RPi. Once I upgrade my NAS, I may try it there. Right now I have it running only on Vera itself. Are you still running it on the Vera?
You can’t run openLuup on the vera, you maybe talking about ALTUI? openLuup is basically a vera emulator you can run on any hardware which then connects to the vera through a bridge and controls it.
Ok then to answer your original question, I just removed it from my vera. I will install it only when I need it as I really prefer it to the Vera UI but I am trying to stabilize the vera so to be on the safe side I am dumbing it down as much as possible to see how stable it can be. The recent rash of report of storage related crashes lead me to monitor this.
Remaining issues:
I am still seeing random luup reloads like most of us albeit not very frequently (once every 1-2 days). I have gone over 15 days in the past without one with a lot of modifications on the vera back on 7.0.20 but it seems the new firmware introduced a few new quirks. It reminds me a bit of the old Windows XP days when the computer could not run more than a few hours without crashing/rebooting but things are gradually improving. (much better than the 1-2 times a day when the vera had internet access)
One event of the lua socket crash out of the luup engine after I changed my firewall settings and I recovered just with a luup reload. I have no idea why. This time it was not caused by my firewall since it can recover even with it on and it seems to be a fluke I cannot reproduce.
The biggest issue I have is I am still having random security sensor triggers (rare, once every few days) and frequent miss of untrip signals from motion and door sensors. They always trip perfectly but often do not untrip. I have notifications setup to troubleshoot this. I suspect it is due to the size of my network. I can actually see the LED on my sensors blink when they transmit, the vera just completely misses them. Again it is random, varies a lot in frequency and affects every sensor type and model I have so it is definitely a controller/zwave network issue.
I have been looking at HomeAssistant, but I’m not sure about it. Still learning, but I’m really no looking to spend a ton of time learning to script different things. That’s the main reason I got away from ZWay. While it can be fun, I have a life outside of trying to turn a light on when a certain set of conditions are met. PLEG handles this fairly easily, as does Rules Engine to some extent.
I am with you. I have learned enough lua now to be dangerous so I don’t want to go spend the time to learn python or javascript. I have better things to do. The learning on home assistant is only to configure it. Even though I had to fix a bug in python on one of the components, overall I am not spending any more time on it.
The main purpose of this endeavor is to make the entire system more reliable. I have been doing some bug fixes on some plugins and openLuup and have gotten close to ideal reliability… except for the vera which still remains the most unreliable element of the system:
It still has random luup reloads every few days I can’t figure out. It now has no plugins but still runs all the scenes which are triggered by zwave devices. How much further do I need to dumb it down?
I still experienced a scene running during daytime while it had a luup.is_night conditional. It is random and not reproducible. I am totally convinced that it is a job completion timing issue inside the vera which I have never seen on openLuup. It could very well be also the reason for the luup reloads
The good news is, over the past week, I have not had any random tripping of devices and have not noticed any failure to untrip so the zwave side seems to be stabilizing.
I’m okay with Python. I have played with a little bit of LUA that I’ve picked up on here. I don’t mind having to do a little bit of programming here and there at all. I’ve got a network of Arduino devices using XBee radios that communicate with a central control program on an RPi3. It’s heavily modified code from Dave’s “Desert Home” project, so it’s not my original code and I take no credit for his hard work. I have heavily modified it and rewritten some portions of it to do what I wanted it to do. I’ve also ported it to Python 3 as Dave is still running Python 2. I did it because I’m going to try to figure out how to integrate it with HomeAssistant…
I created a cron job to run a bash script daily and create a local backup. Just replace the necessary input with yours and then edit your cron jobs. I am doing this on the VM which runs openLuup but it can be done on any computer on your network.
Here is the bash script:
#!/bin/bash
backup_location=#path to your local backup folder#
max_number_of_backups=30 #I want to keep a month's worth of backup#
# Setup environment
. /etc/profile >/dev/null 2>&1
# Go to Vera backups directory
cd $backup_location
# Create a date-specific backup name
export BACKUP_NAME=vera_`date +%Y%m%d`.tgz
# Backup ZWave dongle to Vera (this is async and takes time to complete)
wget -O /dev/null "http://#veraip#/data_request?id=action&DeviceNum=1&serviceId=urn:micasaverde-com:serviceId:ZWaveNetwork1&action=BackupDongle&Restore="
# Wait for Zwave backup to complete (actual time depends on # of zwave devices -- do the timing and adjust)
sleep 60
# Backup Vera (whcih will now include ZWave backup)
/usr/bin/wget -O "${BACKUP_NAME}" http://#veraip#/cgi-bin/cmh/backup.sh?external=1
function number_of_backups() {
echo `ls -1 $backup_location | wc -l`
}
function oldest_backup() {
echo -n `ls -1 $backup_location | head -1`
}
while [ $(number_of_backups) -gt $max_number_of_backups ]
do
rm -rf "$backup_location/$(oldest_backup)"
done
exit 0
Edit:
To complete the instructions in case people are less familiar with linux:
Create a text file above and name it “verabackup.sh” or whathever you want to call it. Remember to edit the ip address of the vera, the number of files you want to keep and the path to the folder where you want the backups to be saved.
Copy that file to a folder you like just remember it’s path on the server you want your backups to be.
Make this sh file executable by sending the command: chmod a+x verabackup.sh from the folder where this script file is
Edit your crontab by executing crontab -e and at the very end of the file add 0 4 * * * /path_to_script/backupvera.sh Don’t forget to replace the path and the name of your script file. This is for the script to run at 4am everyday. You can change this to whenever you want. The first 0 is the minutes and the 4 is the hour. Do not forget the asterisks.
Save the file and you are good to go.[/quote]
The script is returning a 404-Not Found for this:
Backup ZWave dongle to Vera (this is async and takes time to complete)
Testing the http call in a browser also results in a 404 error.
I checked and dongle is not being dumped. I’ve looked through the alleged documentation on the MiOs wiki but can’t find anything useful. The main backup is working correctly and generates the proper .tzg backup file.
