One of my plugins will write new variable values to replace old values about once an hour. I’ve noticed that very occasionally the write will appear to have failed, and it seems to happen when the Luup engine restarts shortly after the variable_set calls. On restart, the plugin will attempt to use a value that should have been replaced by a luup.variable_set call.
My questions are:
Has any other plugin author or scene author witnessed this themselves?
Is there some write cache for luup.variable_set calls, and if so, can I explicitly flush it?
They use an (in-memory) write cache to lessen the wear on the Flash. I don’t remember the auto-flush period, but IIRC they do flush it upon orderly restarts.
There is no published way to force flush this cache.
[quote=“guessed, post:2, topic:179411”]They use an (in-memory) write cache to lessen the wear on the Flash. I don’t remember the auto-flush period, but IIRC they do flush it upon orderly restarts.
There is no published way to force flush this cache.[/quote]
Thanks very much for the reply; that all makes perfect sense, except I think there are “normal” restart situations where at least my variables are not being written out to persistent storage. My log shows that I wrote new variables, but after the plugin restarts, it attempts to use old values that should have been overwritten.
It looks like a bug in my plugin, but all evidence I have suggests that I cannot rely on luup.variable_set in some situations. Just a SWAG, but I think the Luup engine is being overly aggressive in avoiding writing to flash memory too often, meaning that I lose my variable values if the restart happened to soon after I last wrote my variables. I will submit a bug report to the developers; hopefully they will take a look.
If the LuaUPNP crash/restart is happening co-incident with your saving the value (or very shortly there after) in might not get saved to the persistent file.
Is the restart happening because of lengthy Request/Response time from the Nest Server ?
You can validate the issue by looking at the timestamps (if you’re quick) on the user_data LZO file(s) in /etc/cmh/.
As I said, they are supposed to save the contents in an orderly restart (which obv they won’t do in an disorderly/abrupt restart). If you can find cases where they aren’t saving then it’s definitely worth the bug to track.
Thanks for the insights to you both. Unfortunately, I grep’d the log back on January 30th instead of storing the whole thing, so the exact steps that caused the restart are lost. It’s not the Nest plugin that’s having the (extremely rare) issue, it’s the Ecobee plugin. Its OAuth tokens expire about hourly, and every once in a great while, the new ones are lost. I don’t think the restart is due to any kind of comms issue, but again, I lost the log.
Now if the Luup engine stops for any reason, including reasons having nothing to do with my plugin, does this mean that it will lose my unwritten variables if they haven’t been flushed yet? If so, that’s a lousy state of affairs and should be changed by the Vera developers so we can rely on our device variables being what we last wrote them.
I do see that my user_data.json.lzo* has a version written every six minutes (current plus previous five). I notice that it will write a new one when I press the Reload button, but I don’t know if this is at all related to how long the variables are cached.
I don’t remember when they added it, but I believe it was late in the UI4 releases, mostly because they were burning out Flash chips a little too fast.
I fried a Vera2 with flash memory errors at one point
Now if the Luup engine stops for any reason, including reasons having nothing to do with my plugin, does this mean that it will lose *my* unwritten variables if they haven't been flushed yet? If so, that's a lousy state of affairs and should be changed by the Vera developers so we can rely on our device variables being what we last wrote them.
Yes, it's one-out, all out in this case, and all plugins lose out.... since the "store" is a single file, for every Device/State in the system. It's not so-much a cache as just a deferred write. I wouldn't be surprised if it freaks out certain control points also ;)
If your coming across any situation where, under “normal” restart situations it’s not saving then that’s a real problem (obv).
The (Save) operation is going to flush this out before it does it. IIRC you’ll also see that being written in the log files, so you can likely trace from that event to see if it’s doing that in the specific case for the Ecobee. It’s been a few yrs since I’ve looked at that stuff, so I don’t have the specific notes handy.
I never encountered calling luup.variable_set and the variable remaining with the old value, unless LuaUPnP crashed. Look in your LuaUPnP.log to find the LuaUPnP exit code. If it’s 137 or 139 then LuaUPnP is crashing.
Best Home Automation shopping experience. Shop at Ezlo!
