DataMine 2 Vs DataYours

Plugins & Plugin Development Unmaintained
1

Could anyone tell me the key differences between the two?

  • Which one has a lighter footprint?
  • Are both supported by third party Apps like HomeWave(ios) or Imperihome(android)?

I’ve read that Datamine 2 supports (and also recommended) to write the data to an USB drive but no other sources where as datayours pases the data through UDP right?
Both need the AltUI plugin to see the GUI is this correct?

2

Last time I measured, dataMine took about 1.2 Mbytes of disk space and 5 Mbytes of memory when running (it has a LOT of JavaScript, which runs in the browser but which is stored on Vera.) I don’t think it has got any smaller recently. DataYours, on the other hand, is tiny. The total source files are about 66 kilobytes, IIRC. You would be very hard pressed to measure its memory impact with the smallest configuration, but for the standard install it takes a few Mbytes, as does any plugin.

- Are both supported by third party Apps like HomeWave(ios) or Imperihome(android)?

Imperihome supports dataMine but not DataYours.

I've read that Datamine 2 supports (and also recommended) to write the data to an USB drive but no other sources where as datayours pases the data through UDP right?

For either you need, somewhere, for any reasonable amount of recorded data, off-Vera storage space. Otherwise you will bring Vera to its knees by running out of storage. For either, USB or CIFS-mounted network drive will work. However, DataYours does, as you say, use UDP to send data to its back-end which can reside on a totally separate machine to which you only need network access.

Both need the AltUI plugin to see the GUI is this correct?
No. Neither need AltUI, since both have their own UI. However, DataYours can register itself as a DataStorage Provider for AltUI, in which case you can use that UI both to select variables for recording and plot the results.

DataYours is also very modular and can record data into a single database from multiple machines. It also has the capability to plot dataMine recorded channels and saved graphs.

So, you see, they are really rather different animals, although they have much the same aims.

Edit: another key difference is the database structure itself. dataMine has a multi-directory, multi- file structure which grows over time. DataYours uses the industry-standard ‘Whisper’ round-robin database which is of fixed size. So you have to define the maximum time period and sample rate(s) to use, although it has default rules set up to do this for you.

3

Nicely put @akbooer. The key features for dataMine2 for me are:

  • all data is recorded at the best possible accuracy and no data is ever lost.
  • there is no need to configure the frequency with which channels are recorded up front with the risk that the wrong values are chosen
  • The GUI is (for me) very intuitive and user friendly - the focus is on getting the data out in what ever way you like
  • dataMine2 calculates the aggregations at the time the graph is produced where as DataYours calculates the aggregations at the time (or close to the time) the data is recorded. The implication is that with dataMine2 you can do real time analysis and cut the data how you like after it is recorded, whereas in DataYours, you have to rely on your pre-configured aggregations having been correct.

Whilst the footprint of dataMine is quite large I haven’t had any memory issues in operation when storing my data on a USB stick, no have I heard any users on the forums suggesting this is an issue.

Both systems can be accessed remotely in UI5, but AltUI is necessary for remote access on UI7.

I suspect the best one for you will come down to how you intended to use it and what features are important for you.

Can you tell us a bit about your set-up and what you are using it for?

4

I’m planning to use it for recording the values of the ‘Dutch Smart meter’ plugin (it’s a plugin to measure the electricity usage) and creating weekly charts.

I’m on UI7 but I always use VPN to access my Vera so remote access isn’t important for me.
Also the real time measurement is given by the plugin itself.

What is important are the charts it generates. I want to to see (on a weekly bases) if adjustments I made in my power plan has saved energy or not.
My Vera only had 1 USB slot so it’s preferred to transfer the data over the network but I think an USB hub will also do the trick to. I have quite a few plugins installed already so I do have to think about the footprint but haven’t tested the impact if I installed one of the two.
It would be nice to see the charts on my phone (ios/android) but it’s not a must have.

5

I also use dataMine for measuring energy usage (gas and electric) to see how much energy I can save. So far this year I’ve used 11% less energy than the same time last year. dataMine works well on my mobile device especially the save graphs feature which I use in conjunction with the Imperihome app on Android.

It might be worth looking at the documentation available for download here: http://forum.micasaverde.com/index.php/topic,35724.0.html or just try installing it and trying it out.

I’ve attached some sample graphs to give you a feel for it. The gas usage graph shows:

  1. Gas Meter - m3 [Daily Difference] - that’s the difference between the maximum and minimum gas meter readings for the day (i.e. the number of m3 of gas I’ve used in that day)
  2. Heating - Status [Daily On Time] - that’s the number of hours in a day that the heating controller was on for
  3. Temperature (Outside) [Raw Minimum] - that’s the raw/unprocessed data for the outside temperature

You can easily see form the graph the correlation between the outside temperature, the number of hours the heating is on for and thus the amount of gas being used. Have a go and let us know how you got on.

p.s. if the size of the application is a worry, it should be possible to move the bulk of the front end javascript (which is run on the browser but stored on the server) to a USB stick.

6

I’d be interested in comments from anyone else on what features they like most about the two different plug-ins and why they chose on over the other (or perhaps even both!)

@alansmithee - what did you go for in the end (and why)?

7

I want to start with DataYours and see if this meets my expectations.
For now I have tested neither of them because my USB to RJ11 cable was DOA. The next one will be delivered next week.
Will update this post when I have things running

8

Well I’ve got Datayours working for about two weeks.

It’s easy to install and the data generated by the plugin is quite small (228Kb for each device for 5 years).
It nearly didn’t effect the lua startup time what surprised me. For example the pioneer plugin takes longer to startup.

A couple of things what took some tinkering around where;

  • when I selected the variables and interval, I couldn’t adjust it anymore without deleting the old files first.

  • I accidentally mounted a folder to the nas and wrote the variables to that folder without creating a second folder on the nas this caused the files to be writing to the vera itself when it lost connection with my nas.

  • I haven’t figured out if I want the mounting of the nas to be done automatically or by hand. Datamine2 has this option in the the plugin and datayours hasn’t. I’ve read it’s because the automatic mounting can cause a lot of problems.
    While a luup reload doesn’t make the mount undone a restart of the Vera off course does.

  • Making Cifs work took some reading through the forums. For anyone interested I’ve added the steps I followed below

SSH to your Vera and run the this code:

opkg update
opkg install --force-depends kmod-fs-cifs
mkdir /nas
mount -t cifs -o user=[username],pass=[password] //[NAS IP]/[SHARE] /nas

Create a folder on you nas share and use this path in the datayours LOCAL_DATA_DIR variable

I’m very happy with the plugin. Thanks Akbooer

9

Glad to hear it!

It's easy to install and the data generated by the plugin is quite small (228Kb for each device for 5 years). It nearly didn't effect the lua startup time what surprised me.

DataYours has a very small footprint both in terms of memory used and CPU time (also network traffic.)

- when I selected the variables and interval, I couldn't adjust it anymore without deleting the old files first.

You’re right. I keep saying that I will write a utility to do this, although it’s not a easy as you might suppose - it may need to resample the old data.

I accidentally mounted a folder to the nas and wrote the variables to that folder without creating a second folder on the nas this caused the files to be writing to the vera itself when it lost connection with my nas.

Yes, that’s exactly why the recommnedation now is to store your Whisper database on a nas subfolder, eg. /nas/whisper/. That way, if the nas device is not mounted, the subdirectory does not appear and no data will be written.

I haven't figured out if I want the mounting of the nas to be done automatically or by hand. Datamine2 has this option in the the plugin and datayours hasn't. I've read it's because the automatic mounting can cause a lot of problems.

It’s generally USB mounting that causes problems on Vera. CIFS, once you have it set up, is usually reliable. I’ve published elsewhere Lua startup code to check if cifsmount is needed at each reload.

I'm very happy with the plugin.

Again, good to hear - thanks for the feedback.