system
June 19, 2014, 3:28pm
1
Since about a week, my Vera is unusable, lua is restarting repeatedly.
Support has not dialed in, haven’t heard a word from them, so I figured I should ask here, if someone has a tip what to do.
This is the LuaUPnP.log as found in /var/log/cmh/
There are obvious errors in the log, but what to do about it, if related?
Any suggestions welcome, my house is almost unusable as is.
[code]03 06/19/14 17:19:06.010 LuaUPNP: starting bLogUPnP 0 <0x2b447000>
2014-06-19 17:19:06 - LuaUPnP Terminated with Exit Code: 0
---------------exited-------------cmh-ra@fwd7.mios.com cmh-ra@fwd7.mios.com cmh-ra@fwd7.mios.com
Looks like you ran out of memory … need to delete some files … there is not enough room to decompress the State file (which contains all of the information about devices on your system)
system
June 19, 2014, 5:01pm
3
OK, fair enough. But where, and how? I have never really dealt with this kind of /read only fs/shadow/ setup…
This is the output of df;
# df -h
Filesystem Size Used Available Use% Mounted on
/dev/root 4.5M 4.5M 0 100% /rom
tmpfs 62.2M 1.3M 60.9M 2% /tmp
tmpfs 512.0K 0 512.0K 0% /dev
/dev/mtdblock7 11.0M 7.7M 3.3M 70% /overlay
overlayfs:/overlay 11.0M 7.7M 3.3M 70% /
/dev/sda1 503.9M 36.1M 442.2M 8% /tmp/log/cmh
/dev/mtdblock8 4.4M 4.4M 0 100% /mios
Which one is abnormal?
/overlay, but that’s probably a symptom of having user_data.json LZO files that are 6M (expanded or otherwise)
root@MiOS_30eieio:/etc/cmh# ls -ltr *user*
-rw-r--r-- 1 root root 19 Feb 22 14:25 users.conf.timestamp
-rw-r--r-- 1 root root 2 Feb 22 14:52 users.conf
-rw-r--r-- 1 root root 79455 Jun 19 09:25 user_data.json.lzo.5
-rw-r--r-- 1 root root 79406 Jun 19 09:45 user_data.json.lzo.4
-rw-r--r-- 1 root root 79520 Jun 19 09:51 user_data.json.lzo.3
-rw-r--r-- 1 root root 79512 Jun 19 09:57 user_data.json.lzo.2
-rw-r--r-- 1 root root 79638 Jun 19 10:03 user_data.json.lzo.1
-rw-r--r-- 1 root root 79585 Jun 19 10:09 user_data.json.lzo
Go grab the oldest backup, as fast as you can. They only keep them for 7 days on MiOS servers… unless you’ve wired up you’re own to pull them down.
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 4608 4608 0 100% /rom
tmpfs 63728 1172 62556 2% /tmp
tmpfs 512 0 512 0% /dev
/dev/mtdblock7 11264 2592 8672 23% /overlay
overlayfs:/overlay 11264 2592 8672 23% /
/dev/sda1 518320 74420 417572 15% /tmp/log/cmh
/dev/mtdblock8 4480 4480 0 100% /mios
Looks like you have USB logging and a Vera 3
Did you try to restore from a previous backup ? Maybe the file is NOT supposed to be 6MB big ?
Are you using some type of logging plugin (Datamine, DataYours ? ) They could be using the file space as well.
system
June 19, 2014, 5:32pm
6
[quote=“guessed, post:4, topic:181670”]/overlay, but that’s probably a symptom of having user_data.json LZO files that are 6M (expanded or otherwise)
root@MiOS_30eieio:/etc/cmh# ls -ltr *user*
-rw-r--r-- 1 root root 19 Feb 22 14:25 users.conf.timestamp
-rw-r--r-- 1 root root 2 Feb 22 14:52 users.conf
-rw-r--r-- 1 root root 79455 Jun 19 09:25 user_data.json.lzo.5
-rw-r--r-- 1 root root 79406 Jun 19 09:45 user_data.json.lzo.4
-rw-r--r-- 1 root root 79520 Jun 19 09:51 user_data.json.lzo.3
-rw-r--r-- 1 root root 79512 Jun 19 09:57 user_data.json.lzo.2
-rw-r--r-- 1 root root 79638 Jun 19 10:03 user_data.json.lzo.1
-rw-r--r-- 1 root root 79585 Jun 19 10:09 user_data.json.lzo
[/quote]
OK, I see what you mean; :o
/etc/cmh# ls -ltr *user*
-rw-r--r-- 1 root root 504 Apr 22 2010 user_data.json.luup.lzo
-rw-r--r-- 1 root root 19 Jul 16 2013 users.conf.timestamp
-rw-r--r-- 1 root root 17 Jul 16 2013 users.conf
-rw-r--r-- 1 root root 785899 Jun 14 01:46 user_data.json.lzo.5
-rw-r--r-- 1 root root 785863 Jun 14 01:52 user_data.json.lzo.4
-rw-r--r-- 1 root root 785818 Jun 14 01:58 user_data.json.lzo.3
-rw-r--r-- 1 root root 786081 Jun 14 02:04 user_data.json.lzo.2
-rw-r--r-- 1 root root 786109 Jun 14 02:10 user_data.json.lzo.1
-rw-r--r-- 1 root root 786071 Jun 14 02:16 user_data.json.lzo
Go grab the oldest backup, as fast as you can. They only keep them for 7 days on MiOS servers... unless you've wired up you're own to pull them down.
Is it possible to retreive it from outside Vera? I cannot access the backup tab on the Vera. The UI is not working because of the constant restarting…
Login to cp.mios.com and you should see a list of your backup files, along with a way to download them.
I don’t have an account there anymore so I’m going from memory. That said, it could be that this error is coming from the routine that generates the Backups, in which case you have something else that’s triggering it. RTS indicated a “large file”, for example.
system
June 19, 2014, 5:39pm
8
[quote=“RichardTSchaefer, post:5, topic:181670”]Looks like you have USB logging and a Vera 3
Did you try to restore from a previous backup ? Maybe the file is NOT supposed to be 6MB big ?
Are you using some type of logging plugin (Datamine, DataYours ? ) They could be using the file space as well.[/quote]
Yes, Vera 3 and I used to run Datamine, but that was a while ago.
I DID try to restore a backup about a week ago, but that did not help. I think I saw different problems before then, but I did not take the time to deal with it; I simple filed a support ticket.
I can see that I have DM stuff about 2,3M in /overlay/www/dm/* I guess I can remove this now.
system
June 19, 2014, 5:44pm
9
I have found a backup on my computer from beginning of this month, a tgz which is dated June 8.
This one seams ok in size - about 1,2MB?
Is there a guide somewhere how to manually get this back into my vera? I guess I cannot simply unzip it while the vera is up? It contains only ‘etc’.
All this mess started when I had a power failure, but first it was at least running, but it gradually got worse.
In general, you can extract “bits” from the backup files, but first you’ll want to:
a) run Stop_NetworkMonitor.sh
system
June 19, 2014, 6:05pm
11
[quote=“RichardTSchaefer, post:5, topic:181670”]Otherwise I would look to see if there are some new/large files in
Missed that one. No - no big files in /etc/cmh-ludl;
/etc# du -hs cmh-ludl
579.5K cmh-ludl
But /etc/cmh/
/etc# du -hs cmh/*
0 cmh/CONVERT_502SP3_452_MIGRATION
512 cmh/HW_Key
512 cmh/HW_Key2
512 cmh/PK_AccessPoint
512 cmh/cmh.conf
512 cmh/datetime
0 cmh/devices
16.0K cmh/dongle.dump
512 cmh/ergy.conf
512 cmh/ergy.conf.timestamp
227.0K cmh/ergy_cache_paired_devices.conf
512 cmh/ergy_key
0 cmh/first_boot
512 cmh/heal.in.progress
512 cmh/ip_mode
512 cmh/keys
512 cmh/language
512 cmh/language_id
512 cmh/last_backup
512 cmh/last_reboot
512 cmh/last_report
8.0K cmh/network_pnp_sys.xml
5.0K cmh/network_pnp_sys.xml.bak
1.0K cmh/orig
0 cmh/persist
512 cmh/platform
512 cmh/ra_key
512 cmh/ra_password
512 cmh/ra_ports
512 cmh/reupgraded.firmware
1.5K cmh/route.data
0 cmh/scenarios
5.4M cmh/script_output
2.5K cmh/servers.conf
2.5K cmh/servers.conf.default
0 cmh/servers.conf04ctIf
512 cmh/services.conf
512 cmh/srv_timestamp
512 cmh/ui
512 cmh/ui_branding
512 cmh/ui_man
512 cmh/ui_skin
512 cmh/ui_start_page
512 cmh/user_data.json.luup.lzo
768.0K cmh/user_data.json.lzo
768.0K cmh/user_data.json.lzo.1
768.0K cmh/user_data.json.lzo.2
767.5K cmh/user_data.json.lzo.3
767.5K cmh/user_data.json.lzo.4
767.5K cmh/user_data.json.lzo.5
512 cmh/users.conf
512 cmh/users.conf.timestamp
512 cmh/vera_model
512 cmh/version
512 cmh/version_latest
512 cmh/zwave_house_id
512 cmh/zwave_house_id.history
512 cmh/zwave_locale
64.0K cmh/zwave_products_sys.xml
47.0K cmh/zwave_products_sys.xml.bak
512 cmh/zwave_version
script_output :o
After also deleting the datamine stuff, I now have;
/# df -h
Filesystem Size Used Available Use% Mounted on
/dev/root 4.5M 4.5M 0 100% /rom
tmpfs 62.2M 1012.0K 61.2M 2% /tmp
tmpfs 512.0K 0 512.0K 0% /dev
/dev/mtdblock7 11.0M 5.8M 5.2M 53% /overlay
overlayfs:/overlay 11.0M 5.8M 5.2M 53% /
/dev/sda1 503.9M 26.3M 452.0M 5% /tmp/log/cmh
/dev/mtdblock8 4.4M 4.4M 0 100% /mios
So, now I need to figure out what to do the the huge user_data.json.lzo.* files.
[quote=“vespaman, post:11, topic:181670”]So, now I need to figure out what to do the the huge user_data.json.lzo.* files.[/quote]See if you actually have those files @ the larger sizes. The snippet of Log you were pointing to could just have easily been the Backup process itself barfing on “something else” that’s big in the overlay FS.
ie. Run this, and ensure you know why all those files exist (lower the block count as needed)
find /overlay -size +100 -print
system
June 19, 2014, 6:16pm
14
I have them, and they are 10 times bigger than yours. But not as big as the log makes out.
Those are the big ones that I have. They make etc look big;
/# du -hs /overlay/*
6.1M /overlay/etc
360.5K /overlay/lib
34.0K /overlay/mnt
512 /overlay/root
17.0K /overlay/sbin
130.5K /overlay/usr
343.5K /overlay/www
And your find alterrnative;
/# find /overlay -size +100 -print
/overlay/etc/cmh/user_data.json.lzo
/overlay/etc/cmh/user_data.json.lzo.1
/overlay/etc/cmh/user_data.json.lzo.2
/overlay/etc/cmh/user_data.json.lzo.3
/overlay/etc/cmh/user_data.json.lzo.4
/overlay/etc/cmh/user_data.json.lzo.5
/overlay/etc/cmh/zwave_products_sys.xml
/overlay/etc/cmh/ergy_cache_paired_devices.conf
/overlay/etc/cmh-ludl/L_EchoEnergyManager.lua.lzoenc
/overlay/etc/cmh-ludl/D_SmartSwitchController1.json
/overlay/etc/cmh-zwfw/zwave_fw_3.20_jp.bin
/overlay/lib/modules/2.6.37.1/cifs.ko
system
June 19, 2014, 6:20pm
15
…but i guess the user_data.json.lzo contains the much too big stuff that I deleted. If I understand correctly, it tries to decompress about 6,6MB, I still only have 5,2 at hand.
???
system
June 19, 2014, 6:22pm
16
Can i perhaps take the user_data.json.lzo from my backup, do you think? They seam to be good in size, and I have not changed anything that I can think of lately.
(They are about 87KB each)
Run it like this to take the guesswork out:
root@MiOS_300eieio:~# find /overlay -size +100 -exec ls -l {} \;
-rw-r--r-- 1 root root 55895 Nov 4 2012 /overlay/etc/cmh/tmp.lzo
-rw-r--r-- 1 root root 359627 Jul 29 2012 /overlay/etc/cmh/tmp.out
-rw-r--r-- 1 root root 398523 Nov 4 2012 /overlay/etc/cmh/tmp.txt
-rw-r--r-- 1 root root 79484 Jun 19 11:21 /overlay/etc/cmh/user_data.json.lzo
-rw-r--r-- 1 root root 79482 Jun 19 11:15 /overlay/etc/cmh/user_data.json.lzo.1
-rw-r--r-- 1 root root 79536 Jun 19 11:09 /overlay/etc/cmh/user_data.json.lzo.2
-rw-r--r-- 1 root root 79523 Jun 19 11:03 /overlay/etc/cmh/user_data.json.lzo.3
-rw-r--r-- 1 root root 79550 Jun 19 10:57 /overlay/etc/cmh/user_data.json.lzo.4
-rw-r--r-- 1 root root 79560 Jun 19 10:51 /overlay/etc/cmh/user_data.json.lzo.5
-rw-r--r-- 1 root root 65464 Nov 13 2012 /overlay/etc/cmh-zwfw/zwave_fw_3.20_jp.bin
-rw-r--r-- 1 root root 157674 May 14 02:46 /overlay/foo
-rw-r--r-- 1 root root 182839 Sep 25 2012 /overlay/usr/lib/libpcap.so.1.0.0
-rwxr-xr-x 1 root root 658819 Sep 25 2012 /overlay/usr/sbin/tcpdump
As you can see, I’ve left some cruft in there myself
The [tt]user_data.json*[/tt] files ONLY contain the JSON of your config. They’re not the Backup files, so removing stuff from the FS won’t change their filesize (but an errant plugin that writes a TON of State into StateVariables can). These files aren’t typically that big, once they’re decompressed they grow somewhat since they’re pure-text…
system
June 19, 2014, 6:29pm
18
/# find /overlay -size +100 -exec ls -l {} \;
-rw-r--r-- 1 root root 786071 Jun 14 02:16 /overlay/etc/cmh/user_data.json.lzo
-rw-r--r-- 1 root root 786109 Jun 14 02:10 /overlay/etc/cmh/user_data.json.lzo.1
-rw-r--r-- 1 root root 786081 Jun 14 02:04 /overlay/etc/cmh/user_data.json.lzo.2
-rw-r--r-- 1 root root 785818 Jun 14 01:58 /overlay/etc/cmh/user_data.json.lzo.3
-rw-r--r-- 1 root root 785863 Jun 14 01:52 /overlay/etc/cmh/user_data.json.lzo.4
-rw-r--r-- 1 root root 785899 Jun 14 01:46 /overlay/etc/cmh/user_data.json.lzo.5
-rw-r--r-- 1 root root 65272 Mar 1 13:00 /overlay/etc/cmh/zwave_products_sys.xml
-rw-r--r-- 1 root root 232400 Oct 28 2012 /overlay/etc/cmh/ergy_cache_paired_devices.conf
-rw-r--r-- 1 root root 51268 Oct 21 2012 /overlay/etc/cmh-ludl/L_EchoEnergyManager.lua.lzoenc
-rw-r--r-- 1 root root 62714 Dec 2 2012 /overlay/etc/cmh-ludl/D_SmartSwitchController1.json
-rw-r--r-- 1 root root 65464 Nov 13 2012 /overlay/etc/cmh-zwfw/zwave_fw_3.20_jp.bin
-rw-r--r-- 1 root root 355716 Jul 9 2013 /overlay/lib/modules/2.6.37.1/cifs.ko
Make copies of these, and then remove them from your system:
-rw-r--r-- 1 root root 51268 Oct 21 2012 /overlay/etc/cmh-ludl/L_EchoEnergyManager.lua.lzoenc
-rw-r--r-- 1 root root 232400 Oct 28 2012 /overlay/etc/cmh/ergy_cache_paired_devices.conf
Looks like you need some Ergy pesticide, this will at least stop the thing from functioning. You should disable it also, in the Vera UI once you get functional
Make backups of these (off Vera), and then remove them:
-rw-r--r-- 1 root root 786071 Jun 14 02:16 /overlay/etc/cmh/user_data.json.lzo
-rw-r--r-- 1 root root 786109 Jun 14 02:10 /overlay/etc/cmh/user_data.json.lzo.1
-rw-r--r-- 1 root root 786081 Jun 14 02:04 /overlay/etc/cmh/user_data.json.lzo.2
-rw-r--r-- 1 root root 785818 Jun 14 01:58 /overlay/etc/cmh/user_data.json.lzo.3
-rw-r--r-- 1 root root 785863 Jun 14 01:52 /overlay/etc/cmh/user_data.json.lzo.4
-rw-r--r-- 1 root root 785899 Jun 14 01:46 /overlay/etc/cmh/user_data.json.lzo.5
and then restore your user_data.json.lzo* files from the backup. You only need the top one, since the others will regrow soon enough…
Once we get you functional, it would be interesting to see what’s inside the “huge” JSON files, since that’ll be the real cause.
You can email me your compressed user data file … Inca see if I can find and correct the problem for you.
Note this can have passwords or am codes that you enter into any Vera device or plugin.
)