dataMine v0.962 available

MCV have approved the latest dataMine plugin. This version removes the current configuration from the UI5 GUI and adds a new configuration tab to the dataMine GUI. This allows config additional features to be implemented without hitting issues with UI5. I’ve attached a picture of the config screen - it should be self explanatory - select the variables, and edit the data on the right, and click Save. The little pencil icon in the list means that logging is enabled, and the light bulb means that the variable is an energy monitoring variable. Both of these filter up so you can see there is something below when the device/service is not expanded.

The list of channels is slightly different now. You can add and remove columns if you want a different display. Currently this is not saved when you reload dataMine, but I should be able to change this. There are some little icons showing how old the data is - green tick means data is new, yellow is old, and red is older than 1 day. There is also a little ghost icon possible if the device has been deleted in Vera, but the configuration is still active in dataMine. I will add the ability to clean this up in the config screen.

You can now also save channels to be graphed with a single click by hitting the save button. You can then go to “Saved Graphs” to recall the graph. The red X allows you to remove the saved graph (note that at the moment, it’s not possible to update a graph configuration - this will be changed).

I welcome any comments. This change is quite major as the java library was changed to a more flexible system and this should allow me to support more features…

Cheers
Chris

Any reason why the device tile in Vera does not show how many channels are actively recording? It still shows how many channels have been recorded, but the active channels are not there any more.

  • Garrett

[quote=“garrettwp, post:2, topic:173191”]Any reason why the device tile in Vera does not show how many channels are actively recording? It still shows how many channels have been recorded, but the active channels are not there any more.

  • Garrett[/quote]
    Hi Garrett,
    Yep - I removed it. There was method in my madness (but altimately madness prevailed!). The two numbers were previously the channels being actively logged, and the channels that were ‘configured’. In future, I plan on having additional configuration - not just for channels being logged, but effectively extra attributes associated with the variable for the display. This would have tainted the numbers, so (for now at least) I planned to removed the total channels, and just show the logged channels… At least, that’s what I meant to do - but I just looked at the json file, and it’s the total that’s displayed (as you said).

I’m open to suggestions if people want it back - or something else for that matter…

Cheers
Chris

Chris,

It would be nice to know how many channels are actively logging. I can then implement this into AutHomation so when you see the tile, it will show the actively logged channels. Maybe add active channels that have not updated over a certain period?

  • Garrett

p.s. did you get my email with the suggestions for your thermostat and my app and a few other comments?

Hey Garrett,
Yes - I got your email on the thermostats (thanks) - I haven’t had a chance to look at this yet. I was going to see if the subcat could be added to the xml file.

The intention was to show active channels, but I deleted the wrong one. I’ll swap this around for the next release.

I don’t know if it’s possible to embed a web page into AutHomation - if so, it would potentially be relatively easy to add saved graphs… Just a thought :wink:

Cheers
Chris

Hi Chris,

This realy looks great. ;D

Of course we have some remarks/questions.

  • It does not seems to load on my android tablet (chrome), It keeps running on “loading…” (I panic-ed because my faforite tool seems to be broken)
  • How can we remove the “ghost” items? The devices does not exist anymore, I remove the files from /datamine.
  • It looks like th last updated is not always correct, The channel lis report >10 days ago, however I have actual values of today in the graph and files via winscp

Thanks for the good work!

I’m not too sure how to debug this one - I’m using it ok on a Samsung Note-2 (Android with Chrome) and it works great. Let’s park this one for a week (I’m off on holiday from Saturday - sorry) and then have a look at it. I’ll probably need to provide some sort of debug version or something…

This is now implemented, so the next version will have this. It doesn’t (currently) delete the files, but it does remove all traces of the ghost devices.

Yes - this is a “feature” that needs implementing. When dataMine starts (ie when Lua restarts) dataMine doesn’t go through the data to find the last update time, so at this stage, it’s potentially wrong. I say “potentially”, because it depends on a number of things, but generally, it is wrong. I will update this, but it’s not top of the list at the moment…

I’ve found a couple of other small(ish) problems so I will get an update out soon, but it will probably be on the 19th…

Cheers
Chris

Hi Chris

I’ m noticing some strange behaviour on the new version since the auto update. (Using the iPad ios6)

  • When clicking in the link to bring up the reporting window, it now more often than not reports an error communicating with Vera - it’s only worked twice so far after loads of attempts

  • The new graph information shows that some sensors have not been update for many weeks (end of sept) , when i know they have been as the previous version was reporting on them.

  • The main device icon on the dashboard just shows one text line item rather than the previous two, I can’t recall now what the previous two were saying but something about sensors being tracked, and sensors being reported (something like that)

  • The plugin tab has completely gone where I would normally have selected my sensors

  • I have to say the new UI for the graphs is very nice. , would an uninstall and reinstall work, I was not sure what that did with all my sensor data that has been captured over the months

Can you send me a picture of the error - I’m not really sure where/what this is… What happens if you just change the web address from the UI5 (/cmh/) address to the dataMine address (/dm/) - does this do the same thing?

Yes - this was probably the same on the older version, but you may not have noticed it. The last update time potentially gets reset when Lua restarts. The time is only saved periodically, so generally it will show an old time for a while after a restart. Depending on the sensor type, and how often it updates, this may remain for a while. I will change the way this works in the next release.

Yes - as per my response to Garrett, I removed the second one. Unfortunately I removed the wrong one, so I will swap this back. Have a read of my response to Garrett on the “why” of this issue…

Yes - as per the release notes, this version removed the configuration tab from the UI5 GUI and adds a config tab to the dataMine GUI. This is because I can add more config data in this way.

Any reinstall shouldn’t impact sensor data, or dataMine configuration since that is all stored in files on your USB stick. So as long as you configure dataMine to point back to the same USB stick, everything should persist. It looks like only your first point is actually an issue though, so I’m not sure a reinstall is really necessary (or if it would help).

Cheers
Chris

Can you send me a picture of the error - I’m not really sure where/what this is… What happens if you just change the web address from the UI5 (/cmh/) address to the dataMine address (/dm/) - does this do the same thing?[/quote]

Using Chrome on the iPad the URL the plugin links to is http://192.168.1.xxx/dm/index.html and I get the following message “Error Connecting To Vera”, if I keep refreshing the page I eventually get through.

I’ve taken a screen shot and added it below.

If I change it to /cmh/ I go back to the normal Vera UI.

Interesting. This warning isn’t yet implemented on the latest version - it was implemented on the previous version though. So, I’d say that either Chrome is caching the old version (but only sometimes!?!), or there’s some hangup where the previous version of the html and js files didin’t get updated properly.

I’d first try emptying the cache on your iPad - if that doesn’t work, then you could try deleting the /www/dm directory and reinstalling dataMine (this won’t impact your data, or logging or anything like that). I think it’s most likely a cache issue though…

Chris

Excellent tool. I still don’t know what I am doing with Vera and could never get the old datamine to work for me. This one is just so plug and play even I am getting data. My questions…how do I configure datamine to write to usb stick, and how do I look at the usb stick, outside of datamine? (I have been using winscp to look at vera, but still to scared to actually do anything except look for fear of creating a brick.
thanks

Chris is on vacation for the next week. Have a look here to get started on using a usb drive for logging. As for getting the information from the usb drive, your best bet is to still use WinSCP to get the data as removing the drive can cause issue if datamine is still active and the usb drive is in use.

http://forum.micasaverde.com/index.php/topic,7750.0.html

  • Garrett

thanks. followed the instructions. My problem is in winscp where do i find usb stick?

If the USB stick is mounted correctly, then it will simply be mapped to the /dataMine directory. le look in the /dataMine directory and you are looking in the USB drive.

Chris

/dataMine/lost+found/
/dataMine/dataMineConfig.json
/dataMine/InternetOk.log
/dataMine/LuaUPnP.log
/dataMine/mount_tests
/dataMine/NetworkMonitor.log
/dataMine/serproxy.log
/dataMine/signal.flag.log
/dataMine/signal.log

Am I seeing the right files…when I look at contents I don’t see a log of devices?

Hi Chris,

Great tool and thanks for the possibility to save a trend
However if I try to retrieve it I get a screen with a “404 not found” message
I used http://192.xxx.xxx.xxx/dm/graph.html?ref=vermogen
(vermogen is the trend name I gave it while saving)
Am I making a mistake?
I saw in the Vera3 the graph.html file in a directory /dm/app/

Regards,
Jacob

Hi Jacob
Opening graphs using the ref in this way isn’t yet implemented (sorry). You need to load graphs from within the GUI only for now.
The graph. html file is actually from the previous version with the old java libraries, and this will be deleted - probably with the next version…

Cheers
Chris

One thing you did change though is the name of the variable for the datadirectory, it is now called “SetDataDirectory”, as soon as I entered the path to my usb stick (I have mounted this manually without your plugin) everything was running fine again!

Thanks once again for this amazing plugin!!!

One question: I have a device in the graphs which is no more attached to the vera. In the configuration tab the device is not shown any more. Can you tell me how I can remove that device from beeing shown in the graphs tab?

EDIT: Never mind, I removed the according log files and the according lines in dataMineConfig.json. All fine now!

[quote=“crna1, post:16, topic:173191”]/dataMine/lost+found/
/dataMine/dataMineConfig.json
/dataMine/InternetOk.log
/dataMine/LuaUPnP.log
/dataMine/mount_tests
/dataMine/NetworkMonitor.log
/dataMine/serproxy.log
/dataMine/signal.flag.log
/dataMine/signal.log

Am I seeing the right files…when I look at contents I don’t see a log of devices?[/quote]
This doesn’t look quite right. Do you have USB logging also enabled? It looks like it as the logfiles are in this directory. If so, this is bad amd its probably best to disable it for now.

Chris