DataYours on Raspberry Pi

Hi akbooer,

after some tests it seems to work fine with the data stored locally on VeraEdge. I notice that the variable “ChildrenSameRoom” is equal to 1 while I’ve active Cache, Dash, Watcher and Graph.

Then I’ve tried to use a dedicated USB stick connected to VeraEdge with a USB HUB . To USB HUB is connected a USB stick for VeraEdge log and the Serial Current Cost. I got some problems when VeraEdge restart because the dev name changes. Have you any experience with this configuration ?

Alternative to external storage seems CIFS but not working yet (Correct ?).

I’ve a spare Raspberry pi : do you think is possibile to use it for Datayours with the Watcher deamon running on Veraedge and store the data on a storage connected to Raspberry ?

tnks

donato


Edit: The original post above was torn from another thread as it was the first suggestion that DataYours could be run on hardware other than Vera. Full credit to @d55m14 for that. Two things have happened since then:

[ol][li]DataYours has developed further[/li]
[li]The original ‘DataYours on Raspberry Pi’ environment has morphed into a general purpose framework to run any plugin on non-Vera hardware.[/li][/ol]

If you want to run DataYours, then this thread is the one: DataYours - App Store Release - the first post pointed to has a link to the latest (or thereabouts).

If you want to try running any plugin on non-Vera hardware, then openLuup is for you and this is the place: openLuup - running unmodified plugins on any machine

Excellent.

I notice that the variable "ChildrenSameRoom" is equal to 1 while I've active Cache, Dash, Watcher and Graph.
That flag is set to indicate that if you put the master [tt]DataYours[/tt] device into a room, then the children will follow it. They can't be placed in separate room (I couldn't see any reason why you would want to do this.)
Then I've tried to use a dedicated USB stick connected to VeraEdge with a USB HUB . To USB HUB is connected a USB stick for VeraEdge log and the Serial Current Cost. I got some problems when VeraEdge restart because the dev name changes. Have you any experience with this configuration ?
No, none at all. Sorry, can't advise on that. [tt]DataYours[/tt] is specifically designed NOT to mount any devices or do anything special with storage other than access it through a local path. I saw the pain that this caused the erstwhile [tt]dataMine[/tt] plugin and resolved not to go down that path. This definitely falls into the SEP category ("somebody else's problem".)
Alternative to external storage seems CIFS but not working yet (Correct ?).
Indeed, this doesn't seem to install correctly on VeraEdge/UI7. Again, SEP.
I've a spare Raspberry pi : do you think is possibile to use it for Datayours with the Watcher deamon running on Veraedge and store the data on a storage connected to Raspberry ?

This would be fun to do and is certainly possible. There’s two obvious approaches:

[ol][li]Install Lua on the Rpi and run the [tt]DataCache[/tt] daemon on it[/li]
[li]Install a complete Graphite/Carbon system on the Rpi: [url=http://graphite.readthedocs.org/en/latest/install.html]http://graphite.readthedocs.org/en/latest/install.html[/url][/li][/ol]

The whole point of basing [tt]DataYours[/tt] on Carbon/Graphite was that it is an open system, not rooted in any particular architecture, so that you could do exactly this sort of thing.

I've a spare Raspberry pi : do you think is possibile to use it for Datayours with the Watcher deamon running on Veraedge and store the data on a storage connected to Raspberry ?

This would be fun to do and is certainly possible. There’s two obvious approaches:

[ol][li]Install Lua on the Rpi and run the [tt]DataCache[/tt] daemon on it[/li]
[li]Install a complete Graphite/Carbon system on the Rpi: [url=http://graphite.readthedocs.org/en/latest/install.html]http://graphite.readthedocs.org/en/latest/install.html[/url][/li][/ol]

The whole point of basing [tt]DataYours[/tt] on Carbon/Graphite was that it is an open system, not rooted in any particular architecture, so that you could do exactly this sort of thing.

Hi akbooer,

tnks for your reply .

For running your app on Rpi have you any documentation/instructions ? On Rpi do I have to run Graph and Dash daemon too ?

tnks again

donato

I have no instructions, because I have never done it. I have every intention of doing this, though, on my BeagleBone Black board. Assuming that you can install Lua, and the LuaSocket library, then there’s only one routine (the Luup HTTP callback handler) which has to be substituted, and I already have the code for that.

I would have thought that you would want to abandon [tt]Graph[/tt] and [tt]Dash[/tt] outside of the Vera environment. However, [tt]Graph[/tt] is just a subset of the Graphite Webapp and would run just as well as [tt]Cache[/tt] (with the same change.) The dashboard interface, though, is much more intimately connected with Luup and Vera, so is not so easily ported. There are other third-party dashboards and renderers that may, perhaps, also be used.

Without the dashboard interface you could still issue HTTP requests to the renderer and get back CSV, JSON, and also SVG graphics (if the Rpi has access to the internet.)

I’d like basically to store data outside VeraEdge possibly without installing extra module (CIFS for example) on it.

I like the idea of having VeraEdge managed only with the UI7 and plugins so that to have less problem during upgrade . The Dash and Graph module on the VeraEdge will be fine for me.

tnks

donato

Sounds like a great idea - I have a Raspberry PI, so I could imagine taking advantage of this too.

Hi airedale,

I see that in the image of debian for Raspberry there is lua 5.1 already installed.
Do you know how to install the luasocket library indicated by akbooer ?

Tnks

Donato

I haven’t tried it, but here [url=http://files.luaforge.net/releases/luasocket/luasocket/luasocket-2.0.2]http://files.luaforge.net/releases/luasocket/luasocket/luasocket-2.0.2[/url] looks like a good start.

Hi airedale,

I see that in the image of debian for Raspberry there is lua 5.1 already installed.
Do you know how to install the luasocket library indicated by akbooer ?

Tnks

Donato[/quote]

No I haven’t.

Hi akbooer,

I’ve installed lua 5.1 and luasocket on Raspberry. Can I try to install DataCache on it and storing the data on a storage connected to raspberry ?

tnks

donato

Hi akbooer,

I’ve installed lua 5.1 and luasocket on Raspberry. Can I try to install DataCache on it and storing the data on a storage connected to raspberry ?

tnks

donato[/quote]

So sorry, I must have missed this post. I’ll set up a new thread for Rpi, and we can continue the discussion there!

So here we are. What stage are you @d55m14? If I understand correctly:

[ul][li]Lua 5.1 installed[/li]
[li]LuaSocket library installed[/li]
[li]next thing to do is install [tt]DataCache[/tt][/ul]

There is just one Luup call which needs to be replaced (sets up the HTTP callback handler) so I’ll package a replacement and post it here - I’d rather not change the [tt]DataCache[/tt] module at all. What this means is that a small module with just a few Luup call emulations should enable the basic Watcher / Cache / Graph combination to run.

OK, I think I’ve contrived to make [tt]DataCache[/tt] run on a non-Vera machine without any modification of the existing files:

[ol][li]install Lua on target machine[/li]
[li]install [tt]LuaSocket[/tt] library[/li]
[li]copy [tt]L_DataDaemon7.lua[/tt], [tt]L_DataCache7.lua[/tt], and [tt]L_DataWhisper7.lua[/tt] to target machine[/li]
[li]modify the parameter [tt]LOCAL_DATA_DIR[/tt] near the start of the attached file to point to your storage directory[/li]
[li]run the modified file in your Lua environment[/li]
[li]add the RPi IP address and PORT (xxx.xxx.xxx.xxx:2003) to your existing Vera [tt]DataWatcher[/tt] variable [tt]DESTINATIONS[/tt][/li][/ol]

…and you should be up and running.

I’ve not tested this on a RPi, because I haven’t got one, so there may be a bit of debugging needed. I have run it successfully in a test harness on my development machine (iMac.) The attached file simply emulates the relevant parts of Luup, although I’ve not yet included the HTTP callback handler, so the [tt]DataYours[/tt] configuration page will not yet see the cache on the RPi. But it should receive and store data, creating the relevant files and updating them as time goes by.

Would be grateful of any budding beta testers out there. Good luck.

…but I have now tested it on my BeagleBone Black (BBB), and it runs well, taking less than 0.2% of the CPU time. I have it set up receiving data from my Edge and it has created the relevant Whisper database files correctly.

I will add the HTTP listener code so that it becomes a fully-fledged [tt]DataYours-Graphite[/tt] daemon.

The ease of doing this, and the reliability (so far) of the BBB means that I am wondering just how much of the Luup environment I can/should emulate in order to port more of my plugins to this platform.

Just a snippet of the running log and created files to prove it:

Sun Mar  8 14:41:35 2015, call: UDP_listener_2003, cpu = 1.080sec (0.155%)
Sun Mar  8 14:41:36 2015, call: UDP_listener_2003, cpu = 1.090sec (0.157%)
Sun Mar  8 14:41:37 2015, call: UDP_listener_2003, cpu = 1.090sec (0.156%)
^Clua5.1: Test_Cache.lua:76: interrupted!
stack traceback:
	[C]: in function 'select'
	Test_Cache.lua:76: in main chunk
	[C]: ?
root@beaglebone:~# ls -l
total 112
-rwx------ 1 root root  6208 Mar  7 22:29 L_DataCache7.lua
-rwx------ 1 root root 15417 Mar  7 22:29 L_DataDaemon7.lua
-rwx------ 1 root root 27340 Mar  7 22:29 L_DataWhisper7.lua
-rwx------ 1 root root  2544 Mar  8 14:29 Test_Cache.lua
-rw-r--r-- 1 root root  6132 Mar  8 14:40 Vera-45101161.005.urn^akbooer-com^serviceId^EventWatcher1.AppMemoryUsed.wsp
-rw-r--r-- 1 root root  6132 Mar  8 14:40 Vera-45101161.005.urn^akbooer-com^serviceId^EventWatcher1.CpuLoad05.wsp
-rw-r--r-- 1 root root  6132 Mar  8 14:40 Vera-45101161.005.urn^akbooer-com^serviceId^EventWatcher1.MemAvail.wsp
-rw-r--r-- 1 root root  6132 Mar  8 14:40 Vera-45101161.005.urn^akbooer-com^serviceId^EventWatcher1.MemFree.wsp
-rw-r--r-- 1 root root  6132 Mar  8 14:34 Vera-45101161.019.urn^upnp-org^serviceId^TemperatureSensor1.CurrentTemperature.wsp
-rw-r--r-- 1 root root  6132 Mar  8 13:54 Vera-45101161.021.urn^upnp-org^serviceId^TemperatureSensor1.CurrentTemperature.wsp
-rw-r--r-- 1 root root  6132 Mar  8 14:39 Vera-45101161.047.urn^akbooer-com^serviceId^DataYours1.AppMemoryUsed.wsp

Hi akbooer,

I’ve tried Datacache on Rpi and the messages seems ok (I’ve attached a log).

The only error message is :

sh: 1: GetNetworkState.sh: not found

Have you any idea ?

tnks donato

Yes. That error is because the mechanism used on Vera to recover the IP address of the machine is not available on the RPi. It generates an error, but then goes ahead and uses the 127.0.0.1 IPv4 loopback ‘localhost’ address, which is fine for the purpose.

So what next?

[quote=“akbooer, post:16, topic:186265”]Yes. That error is because the mechanism used on Vera to recover the IP address of the machine is not available on the RPi. It generates an error, but then goes ahead and uses the 127.0.0.1 IPv4 loopback ‘localhost’ address, which is fine for the purpose.

So what next?[/quote]

I’ve intended you have to add some code to Rpi to access the data from DataDash and DataGraph on Veraedge.

Is it correct ?

tnks

donato

[quote=“d55m14, post:17, topic:186265”]…you have to add some code to Rpi to access the data from DataDash and DataGraph on Veraedge.

Is it correct ?[/quote]

Possibly. I take it that your storage on the RPi is not visible from Vera (you didn’t want to use CIFS)? So I need to check that [tt]DataGraph[/tt] can access [tt]DataCache[/tt] remotely with some other mechanism (native Graphite cache access.)

Well, we are making some progress, anyway!

PS: I’d be interested to know the asymptotic value of the cpu percentage that you get if you leave things running a while on the RPi.

Hi akbooer,

Yes i’d like to install less extra software as possible on VeraEdge .

Porting datagraph on Rpi too will simplify the remote access of datadash ?

Ps: i’ll measure the cpu Usage of datacache running.

Donato

Yes, in fact I’ll take a look at moving [tt]DataDash[/tt] there as well. I have just discovered that the BeagleBone Black comes with CIFS ready installed (not sure if this is so for the RPi) so it’s trivial to have the Whisper database sitting on a NAS.

Ps: i'll measure the cpu Usage of datacache running.

You might take a look at the memory usage too… it was a delight to see enormous amounts of free space available - quite a difference from Vera.