dataMine 0.979

I think you should easily be able to format the drive yourself, and get the full capacity. It’s just that the script that formats the drive for USB logging limits the partition to 500MB. Formatting it to get the full capacity should be achievable - you would just need to do it manually.

Cheers
Chris

Hello, a long time that I didn’t used the plugin, seems great now ! Anyway, I have not seen all the pages of the topics, sorry, but now my problem is when I try to log and graph the wind speed, I just have a line with the actual value (see picture to understand). Bad parameter somewere?

Hi,
Have you just started logging this variable? If so, this may be normal - the weather plugin only updates every hour or so I think, and dataMine will show 1 days data by default, and it will use the current value as the starting point in the absence of more data (resulting in the straight line).

If you’ve run this for a while already, please let me know, otherwise I would give it a few hours to make sure the data has updated and see if that helps.

Cheers
Chris

Hi, Chris: I’m still getting problems. So I upgraded to .622 firmware for my Vera Lite. Installation of dataMine went fine. But still no data was plotted or saved. The top right Online Status shows red. I pulled the log (see below) and it seems that dataMine cannot open the file 2265.txt among other errors. But I am using a 2GB freshly formated (fat32) drive and it shouldn’t be full. When I plugin that USB drive in my laptop I saw these files on it (see attached pic). Can you help me again?

Btw I have been installing/uninstalling dataMine in the minutes leading to the error in the log so there might be weird behaviors in the beginning. Also the USB drive is plugged in to a USB hub, along with the USB RFXCOM device because there’s only 1 usb port on the Vera Lite. I have tried two different USB hubs, powered and un-powered, 4 different USB sticks, FAT32 or EXT3 over the last few days to see the same error over and over again.

Hi Chris, thanks for your answer, yes, I’ve just started logging this variable. Nw it’s ok, I’ve just missed one point.

As I store Vera logs in USB, I’ve forgotten to set the following parameters: SetManualMount to 1

Anyway, as you can see, it’s always important to read the documentation !!! :wink:

Table 2: Suggested configuration for use with Vera USB logging.
Configuration Variable Suggested Value
SetDataDirectory /tmp/log/cmh/dataMine
SetMountPoint Not required to be set
SetManualMount 1
SetMountUUID NA

[quote=“Chris, post:21, topic:175538”]I think you should easily be able to format the drive yourself, and get the full capacity. It’s just that the script that formats the drive for USB logging limits the partition to 500MB. Formatting it to get the full capacity should be achievable - you would just need to do it manually.

Cheers
Chris[/quote]

Sorry for the delay in responding. Been dealing with some stuff offline.

So I got an Ubuntu box up and running (if it matters, it’s a Parallels virtual machine running on top of OS X), and was able to access and format the remaining ~39.5GB of the USB drive and labelled it “dataMine” (ext3). I didn’t touch the Mios partition. Rebooted Vera with the drive attached and relaunched dataMine.

I forced dataMine to treat this as a new install and it gave me the option of selecting the “Mios” partition or the “dataMine” partition. Unsurprisingly, I chose the “dataMine” partition. After following the prompts to reload UI5, I went back and restarted dataMine. It presented me with the select UUID dialog again. Interestingly, neither partition showed up at that point. I went to USB logging in UI5, and the box was unchecked. So on a whim, I checked it and ran through that process–but I got a FAILED message. Now neither partition shows up in UI5 or dataMine. Hook the drive back up to the Ubuntu box and both partitions show up as clean and are mountable.

  1. Reboot Vera without drive attached and then reattach it.
  2. Run dataMine debug. Both partitions show up with no errors.
  3. Load dataMine, and it seems happy as a clam.

Not sure what happened during the dataMine reload process (after adding the new partition information through the dataMine GUI), but a power off reboot of Vera (with no drive attached) and then attaching the drive after Vera has fully booted seems to have solved the problem for dataMine. However, now Vera says that USB logging is turned on, but the drive is not mounted.

  1. Reload UI5, and now USB logging is unchecked and it says that the drive is not mounted.
  2. Load the dataMine debug and nothing shows up under -2-.
  3. SSH into Vera and confirm that the USB drive is in fact not mounted.

Now it appears that dataMine is saving it’s files to Vera flash memory.

Sorry for the verbose logging ;D, but I wanted to be thorough in my description. I feel that I’m close, but I’m not there yet. It could be because Vera doesn’t like a large USB drive directly attached. Will start from scratch and may try a USB hub next. May be a couple of days before I can report back on progress. I haven’t given up on this yet.

Dave

Hi,
Sorry for the slow response - I’ve been abroad for work without much connectivity…

@DaveL17
You will need to restart Lua or Vera after initial configuration (ie selecting the UUID in the GUI). This is to allow dataMine to remount the drive (since I don’t know of a nice way to force a reload).

Other than that I’m not sure what’s up. I can suggest the following -:
Restart Lua (hit the reload button in UI5)
Wait for a minute or so
Click on the debug link in dataMine device
Post me the output (either here, PM or email). I’ll take a look and see if I can spot anything.

Cheers
Chris

@nanotube
So sorry for the delay in responding to you - I’ve been away for work :frowning:

Are these the only files in the directory? If so, then this may explain the issue - there is no database directory for some reason. Try creating a database directory in this directory and see if that helps. You would also need to disable/enable logging for the variable again. I’ll take a look at the code tonight and see if I can see why this directory hasn’t been created since I think it’s in the startup…

Also, if you can send the dataMine debug file from the debug link in the dataMine device (if it works - it should as dataMine seems to be running). This provides the log file like you sent, but also some other configuration data which is useful to see what’s happening.

Cheers
Chris

Nothing to apologize for my friend.

Yes. Have restarted Lua many times. The closest I can get on the debug page is that when the problem is occurring, nothing is mounted. I can repeat the process–I restart Vera with just the USB logging stick installed. Then after the reboot is complete, I can plug in the dataMine stick and get it running. But the next time Lua restarts, both sticks are “gone”. Rinse and repeat.

I’m nearly 100 percent certain that it’s nothing to do with dataMine whatsoever.

This weekend, I’m going to undo USB logging under Vera (I don’t have that many devices anyway) and use USB exclusively for dataMine, and see where that leaves me. I suspect that I’ll be fine after that.

Thanks for always being there Chris.
Dave

@nanotube
So sorry for the delay in responding to you - I’ve been away for work :frowning:

Are these the only files in the directory? If so, then this may explain the issue - there is no database directory for some reason. Try creating a database directory in this directory and see if that helps. You would also need to disable/enable logging for the variable again. I’ll take a look at the code tonight and see if I can see why this directory hasn’t been created since I think it’s in the startup…

Also, if you can send the dataMine debug file from the debug link in the dataMine device (if it works - it should as dataMine seems to be running). This provides the log file like you sent, but also some other configuration data which is useful to see what’s happening.

Cheers
Chris[/quote]

Oh Chris, you are so lovely. Are you Canadian or possibly a senior lady? I am taking your precious time out of your important job and hobbies, I’m definitely not one to say sorry to.

And indeed all I see were those files, no folder or the “database structure” on the USB drive. I have no idea how to create said database structure though. I am not using USB logging in Vera. The drive was cleanly formatted w/FAT32 in a Win7 machine. Here I’ve attached the debug file. Thanks again!

@DaveL17,
Do I understand correctly that the debug link is not working? Are you just getting an error if you click on the debug?

USB support in Vera isn’t great, and USB logging definately makes things worse. I’ve dropped the use of USB now and am using a share on my Linux server using CIFS. The next version of dataMine will support this - it seems to work well, and is no slower than USB…

Chris

Ah - nope. Kiwi/British - originally from New Zealand, but been in the UK a long time :wink:

Ok, so I would say that the database directory is not there, and this is probable the issue. I don’t understand why this isn’t being created since it should be generated each time dataMine starts (Ooops! Just found the problem!).

To fix your problem, cut and paste the following into UI5. Go to the APPS tab, then click DEVELOP APPS, then click TEST LUUP CODE (LUA) and paste in the following lines and click GO. Hopefully this will solve your problem.

os.execute("mkdir /dataMine/database/") os.execute("mkdir /dataMine/database/1/") os.execute("mkdir /dataMine/database/1/raw/") os.execute("mkdir /dataMine/database/2/") os.execute("mkdir /dataMine/database/2/raw/")

Cheers
Chris

[quote=“Chris, post:31, topic:175538”]@DaveL17,
Do I understand correctly that the debug link is not working? Are you just getting an error if you click on the debug?[/quote]

Hi Chris - Sorry, I should have been more clear. The debug page works just fine. Sorry for the confusion.

What I meant was, when the error occurs, I would load the debug page and it would report that neither the dataMine nor the USB logging stick were mounted. In other words, the ‘-2-’ section was completely blank. I was also able to confirm via the ‘mount’ command under the shell that nothing was mounted.

I have just restarted dataMine with USB logging turned off. Will report back on my progress. As I said, I suspect that things will run smartly.

Thanks again,
Dave

Ah - nope. Kiwi/British - originally from New Zealand, but been in the UK a long time :wink:

Ok, so I would say that the database directory is not there, and this is probable the issue. I don’t understand why this isn’t being created since it should be generated each time dataMine starts (Ooops! Just found the problem!).

To fix your problem, cut and paste the following into UI5. Go to the APPS tab, then click DEVELOP APPS, then click TEST LUUP CODE (LUA) and paste in the following lines and click GO. Hopefully this will solve your problem.

os.execute("mkdir /dataMine/database/") os.execute("mkdir /dataMine/database/1/") os.execute("mkdir /dataMine/database/1/raw/") os.execute("mkdir /dataMine/database/2/") os.execute("mkdir /dataMine/database/2/raw/")

Cheers
Chris[/quote]

That’s it - it works! Thanks.

I did it and it that sort of worked because I then got the following errors:
dataMine: Unable to open file for write /dataMine/database/5/raw/2267.txt <0x2c16f680>
Error: ‘/dataMine/database/5/raw/2267.txt: No such file or directory’

So I created that as well - added a whole lot of things to log and got to directory 10
… I’m hoping this is not a long term continuously incrementing number :slight_smile:

[quote=“rjwilson01, post:35, topic:175538”]So I created that as well - added a whole lot of things to log and got to directory 10
… I’m hoping this is not a long term continuously incrementing number :-)[/quote]

@rjwilson01: I believe dataMine creates a directory for each variable tracked. I’ve found that it might take a little while for the text file to be generated (it may be that it’s not generated until the tracked value changes.) I have a light sensor that took a fair bit of time to show up (because I created it at night.)

@Chris: after shutting off Vera USB logging and having only dataMine hitting on the USB port, things seem to have settled down. I haven’t done a power off reset yet (I’m hesitant because things are working well and I have Vera on a battery backup).

[quote=“Chris, post:32, topic:175538”]To fix your problem, cut and paste the following into UI5. Go to the APPS tab, then click DEVELOP APPS, then click TEST LUUP CODE (LUA) and paste in the following lines and click GO. Hopefully this will solve your problem.

os.execute("mkdir /dataMine/database/") os.execute("mkdir /dataMine/database/1/") os.execute("mkdir /dataMine/database/1/raw/") os.execute("mkdir /dataMine/database/2/") os.execute("mkdir /dataMine/database/2/raw/")

Cheers
Chris[/quote]
Hi,
Thanks, it also solved my problem. It was a fresh install on a new Vera with last firmware and only a few Zwave device.
And also thanks for this plugin.