Memory Vera plus

Greetings,

LAst couple of weeks I had an issue with virtual switches which didn’t wanna toggle anymore. After tech support looked at my system they said I was running out of memory and they freed up some memory.

Since I also installed Datamine2, I log the memory now as well since last week.

But how bad is this for a vera plus?

-every day around 0300 I do a scheduled reboot ( I see on the graphs the “memory free” jumps up at that time).
-but during the day the memory used goes up.

Do I need to get a second vera plus and balance the load?, do I have to many devices/scenes/PLEG?

Thanks,
Cor

Hey Cor,

I think your plugins and virtual switches are the largest culprits. I have watched the virtual plugin threads over a few years and their memory use versus a standalone straight z-wave device is vastly different.

There have been recent topics on ways to flush memory without rebooting but you might have to do some experiments and I’m not sure of the side effects. I would suggest determining a few virtual switches and plugins you can live without and check your stats again.

@ Tomtcom:

I have read the rebooting thread as well , that hinted me on using datamine to log the memory usage, so I have a bettert overview.

The graphs just now shows the same as yesterday , after the scheduled reboot ( at 3:15) memory free goes up quite a bit and memory used goes down.

I do not want to get rid of devices I have , als not scenes/PLEG , virtual switches etc. I even want to expand a bit more, my house is still not automated enough ;D

So the question remains , is my vera plus “full” , do I need to get a second one if I want expand , or is it not that bad “yet”.
Or do I even have to get a second one and balance the load a bit of what I have now?

Thanks,
Cor

Fist, check if your Datamine2 plugin really saves the data to the USB stick or to the internal memory (you can tell it going to the Datamine2 settings (“Datamine2” section) and checking if amount of total memory matches the size of USB stick or Vera internal memory.
Also make sure that you have local saving of Vera logs disabled. This doesn’t go well with Datamine2, at least if you save data from both sources to the same stick. They are how-to’s about saving logs from Vera and informations from Datamine2 to separate sticks or to separate partitions from the same stick, but I didn’t try it. Most of the time I have only remote access to my Vera, so I can’t access local Vera logs anyway.

Then you can check your plugins, as they are most probable source of data consumption (at least that’s what Customer Care told me).

In normal operation, Vera should clear the cache by itself on regular basis (every 1-2h), and the graph should look like in attached picture.
I don’t know how many devices/plugins you have so I can’t judge if your Vera is overloaded or you have some misbehaving device/plugin. I have about 40 nodes (120 “devices”) and 6 plugins and my system if fine, except occasional memory drains (i.e. once per month).

@ Kwieto, Thanks for your reply.

I have a 8GB USB stick in the vera plus , and attached a screenshot , it looks like Datamine sees the 8GB stick.
BUT , I also attached a second screenshot where you tell me to check “Also make sure that you have local saving of Vera logs disabled” , but when I check the logs section ( see attached screenshot) it says :"USB statusDisabled Not mounted " What is going on?

And which one do I uncheck?

edit and this is what is on my vera plus:
-16 plugins
-64 physical devices, from about half of them have child devices. (fibaro double relais and universal sensors with temp probes)
about
-20 virtual switches
-about a totall of 160 devices
-44 scenes
-6x PLEG: shitloads of conditions between 10 and 30 per PLEG instance.

There is one plugin I don’t use ( Sercomm IP Camera) but I am unable to remove that one somehow.

Thanks,
Cor

That is OK.
If you have local logging enabled (checked “store logs on USB device” option) then Vera erases USB stick to its format (it is no longer readable by computer).
If you don’t have it enabled, then you’ll see “Disabled not mounted” as a USB status. Datamine can access the stick anyway.

Seeing list of your devices, I would first try to decrease number of plugins (you have a lot of them) and Virtual Switches (maybe replace at least part of them with multiswitch, or variables within scenes? Do you really need that many? As far as I remember each virtual switch consumes about 2MB of space, 20 of them is about 40MB which is almost 1/5 of your total Vera Memory)
I would also consider reducing number of PLEG instances: I’ve read some comments that overuse of PLEG can affect your system speed, so after my first controller got bricked I decided to avoid installing it untill it will be really necessary (same logic for most of other plugins I’ve had before).
For now I use Ping sensor, Netatmo weatherstation, Datamine, System Monitor, 2 virtual switches (one can be probably replaced by global variable) AltUi (mostly to have access to datamine remotely), HouseModes and VirtualPanel (installed for testing, but probably will remove it as I don’t have time to dig in it) plugins.
Second controller has even less of them (Netatmo, AltUI, System Monitor, Datamine and Virtual Rain sensor)

@ Kwieto: I have been looking at my system. I have again tried to remove the “sercom Ipcamera plugin” but it comes back every time. Also the “alexa helper” plugin I will try to remove.

For virtual switches , there are 3 installed which I can remove And PLEG instances ( I have a totalll of 6 PLEG) 1 I can remove , and there is one Test PLEG. Which I can remove if really neccesary.

If I remember well , the multiswitch have 6 or 8 switches , that is much better as 6-8 virtual swicthes? Because , that I can easily change ( If Imperihome can handle them :slight_smile:

To be sure: With a PLEG instance is meant how many time is is installed as a device? , not the amount of conditions in the PLEG?

Also make sure that you have local saving of Vera logs disabled.
Do I need to uncheck here anything (see attachment) ? , because I see there also "Lock Log Levels (not recommended)" is checked in my case, all options are checked. :o

It looks like I will havr to install a second vera , although I still have a vera light unused here… ???.. choices choces choices…

edit: after ca 20 minutes a luup restart and I see the “Sercomm IP Camera” has installed itself again ???

Many thanks for you help,
Cor

This is known issue, I also have these plugins and they can’t be removed. Didn’t count them as I consider them as a “bug” I have no use of them

Maybe you can replace some of them with variables stored within a scene, or global ones? I can’t give you really advice since I don’t know what you use them for.

As far as I understand there is some “minimum” amount of memory which have to be used for each device, so 1 multiswitch (capable keeping 6-8 “switches”) use similar amount of memory as 1 virtual switch.
Changing 8 virtual switches to 1 multiswitch should free amount memory equal to removing about 7 virtual switches (let say 7 x 2MB = 14MB).

I’m not an PLEG expert, but as far as I understood from reading here and there, both are important.
As I mentioned above, I took general approach to keep my system lean (which includes: as little plugins as possible), so when I weighted profits (easy managing “and” conditions, continuing schedules in case of luup reloads) and loses (at least two plugins needed to be installed: PLEG and PLC) I decided that for those couple of scenes where it would be useful I can live with writing in lua. If necessary I split scene in two (like: if something has to be started at X and finished at Y I have one scene for “ON at X” and second “OFF at Y” instead of making a schedule for a duration between X and Y).
I also manage many things via Home Modes, which brilliantly cover most of my needs for virtual switches (though I have couple of scenes which use home mode AND state of the virtual switch)

No. you can disable lock log levels and verbose logging as by default these options are disabled, but I left these options checked after support requested me to do this when they were working on some issue couple of moths ago.

Or you may reconsider using of some tools you have. I wonder for example why do you need as many virtual switches, as in my case I have only 2 of them, and one could be in fact changed into global variable. I use it to run a geofence based scene, to track the direction I am moving (so the scene is triggered only if I move forth, but not back the way). Since the switch changes state for very short period (like: 5-10 minutes) I can set a global variable and change its value for this period, as the risk of reboot or luup reload in such short time is really low.
The other is used to track more permanent condition (if I have guests at home).

@ Kwieto , Thanks for your big reply.

I need most virtual switches , I use them for example for:

  • Every Family member has a virtual switch to switch the security system on or off , switching it on or of sends a message to everyone , saying that person deactived or actived the system.
    -switching the heating system on and off , this is for PLEG , which activates lots of other sub systems.
  • other systems which I switch on and off via PLEG , via virtual switches.

I will look if I can change some to the multiswitch.

Lua is unfortunately not for me , PLEG is very difficult as well , but at least there is some logic in there for me.

No clue how to work with global viariables… I even don’t know what they are ???

At least I know nothing is wrong with my system ( probably) , but most likely overloaded.

The following days I will take a look into multi switches and remove the virtual switches I don’t use. I can also delete some scenes.

The other think which is a bit wierd in my system is that when there is a LUA restart , the DATAmine2 takes ages to load: "dataMine 2 graphing [307] : Running Lua Startup " something like 15-20 minutes mosty of the time. During that time I notice that virtual switches can not be toggled anymore.

Also I will consider adding a third controller in my system.

Thanks,
Cor

You can use geofencing for that, but you will loose ability to be informed who entered home.

Isn’t the heating system turned on/off by some z-wave switch? Then why using separate virtual switch instead of just monitoring state of that z-wave switch?
I steer my heating with house modes and it works pretty well. If you have thermostat devices, you can set certain state (like Heat/Cool/Off/Auto) and temperature to be set when certain house mode is activated. If your heaters work as simple on/off devices, you can make specific scenes to be run according to the certain modes. For more fine-tuning scenes can be used as well (like: set certain temperatures for specific time periods).
I have two locations: one has central (hot water) heating so I have z-wave valves installed on each radiator. I have two scenes, one set lower temperatures for each valve at certain time in the evening, second sets higher temperatures in the morning. The rest is steered by house modes (i.e. lower temperatures when I’m away or on vacations).
At second place I have electrical heaters which work basically as on/off devices. Then I have temp sensor in each room and two scenes for each heater: one turns it on if temperature drops below certain level (different for different times of day), second turns it off if temperature increases above defined level. Scenes work in house mode home or night. For “away” and “vacation” modes I have similar duo of the scenes respectively, but with different temperatures.
Now I experiment with Qubino flush ON/OFF thermostat, which works as ON/OFF switch, but the only thing you need is to set desired temperature (works similar as radiator valve). It can be steered via house modes as well (i.e. "keep temp 15C if I’m away, turn off when I’m on vacation, etc)

Similar question as above, can’t you use states of physical z-wave devices for that?

I found Lua less difficult than PLEG as it uses typical logical operators and you have everything (conditions, triggers, logic, actions) in one place while PLEG organizes them in separate sections which makes it more difficult for me to orient which is for what.

You can define variable as local (to be used only within certain scene) or global (accessible from other scenes). For example you may define variable “X” to store status of a switch (or any other data you define), to be used as a condition (if X == n then…) or to be replicated to other devices (i.e. "check the stat of switch 1 and then set states of switch 2, 3, 4 to match it)
If variable is defined as global one, you can use it in various scenes, just referring to its name. If it is local, works only within the scene where it is defined.

I don’t think overload is the only issue. If you’ve just had too many things, you would expect your amount of free memory to be very low and/or slow speed of your system. But amount of free memory should be more or less stable at certain level, while in your case it decreases continuously. This suggests that something in your system is not right and too many things is not the only trouble.

Then maybe something is wrong with datamine plugin settings, or the USB stick to which datamine saves data. In my case if it takes seconds for such message to dissapear, that’s all. And you have only 28 channels monitored, while I have 110.

[quote=“Cor, post:9, topic:198791”]snip

Also I will consider adding a third controller in my system.

Thanks,
Cor[/quote]

I’ve never heard of anyone adding a third controller as another Slave. Can it be done?

Mmmmmm, a Vera threesome… :-*

Stil trying and looking what I can delete ( or change) and it isn’t much.

Attached a screenshot from the memory the past 24 hours.
I see there totall memory , which is the top red Line.
memory available, the green line.
Memory used: Blue line.

Abouth half in the day , the memory used goes above the memory available, but still under “totall memory”

How do I interpret this?
Around 14:00 it uses more memory than there is available?? is this correct? because that would be very very bad I guess. Although it seems my system is very stable . Almost no luup restarts ( maybe a bit on the slow side.)

What is your opinion?

Thanks,
Cor

No. Memory available partially overlaps with memory used parameters
Calculating roughly, memory free + memory used = memory total.
Rougly, because your graph miss at least two additional parameters: cached memory and memory buffers.
If you add these, then you’ll see that memory available contains memory cached so it is partially overlapping with memory used (the logic behind is that memory available = “freeable” and cached memory is able to be freed by clearing process).
For tracking how your system is loaded and how much memory left, free memory is a better parameter to track.

Just a thought, have you tried putting Vera on its own subnet/vlan off you router?

A couple of years ago when I still had my VeraLite, I was having constant memory exhaustion and subsequent LUUP reloads. I tried for months to track down what is draining the memory over time such as plugins, logging, etc. On a whim, I just decided to put the vera on its own isolated subnet away from the DLNA and other broadcasting going on with the dozens of other things on my network. This fixed my issue and even since migrating to the Plus, I have kept it on its own vlan with just another device and my unplanned LUUP reloads only happen every few weeks.

I too have dozens of zwave, IP based, and cloud connected devices integrated , with about 15 plugins, and a decent amount of PLEG logic including timers.

In my opinion something is wrong here beyond simple overload by plugins and devices.
I’m not an expert, but “common sense” logic suggests that the more plugins/devices you have, the lower amount of free memory you would expect in general, but it should stabilize at some level.
Of course minor changes are possible, but if everything work I would expect minor jumps/drops around a certain level (in my case it is within 10-20MB corridor, with occasional jumps for 30-40MB). Most of the jumps are related to the amount of cached memory, but it may happen that amount reserved for memoryBuffers (whatever hides after that) also increases.

But here the line goes down and down, so something is stuffing it probably more than it is necessary.

Tried some troubleshooting.

I disabled all my scheduled scenes , most of them have luup code in them. Including a scene which reboots my vera plus at 3:20 in the morning.

After a day my system was completely unresponsive. I checked the datamine memory and the memory used was almost touching the red line.
( please see attached screenshots).

After a reboot and re-instating the scheduled reboots , all was “normal” again.
Can I conclude nothing is wrong with my scenes which have LUUp code in them?

What would be a next step for troubleshooting?

Deleting plugins? that will be a mess than , since all the plugins installed I use in scenes and PLEG.

Thanks, Cor

I don’t see any other solution to track-down the problem than disabling plugin by plugin and checking if that helped, or disabling them all at once and then re-installing one after another.
Or you can leave your system as it is and rely on the daily reboots/clearings, but I wouldn’t recommend that for long run.

Tried putting your Vera on a separate subnet/vlan/network than where the rest of your devices are ? Allow your firewall to route between them, but isolate the broadcast domain ?

@ niharmehta: I unplugged the vera plus from the internet/local network. no change.

@kwieto: That is going to be something, since all my plugins are integrated in PLEG and scenes. If I delete a plugin , what will happen with the PLEG instance and scenes? : :-\

I can not find am option to disable plugins , just uninstall them.

Cor