DataSync for ConstantGraph - logging and graphing

Plugins & Plugin Development General Plugin Discussion
#1

Hi all,

It’s been a few years since I worked on the dataMine 2 logging and graphing plug-in. In the meantime I worked on dataMine 3 which moved in a different direction and so for the latest iteration I decided to start from the ground up. As such, I’ve created a logging and graphing site https://www.constantgraph.com and a plug-in to send data from Vera to the site for analysis and graphing. I’ve submitted the plug-in for approval so hopefully it should be available soon. I’ve made it backwards compatible with dataMine so you can synchronise your graph configurations and entire data history (I’ve got several years worth now) or monitor independently if you don’t use dataMine.

In the meantime, I’m looking people willing to beta test the system and suggestions of what features might be useful. Anyone interested?

Thanks

1 Like
#2

Happy to test this! I’ve been trying some bits with influx and Grafana, however it gives me another VM to manage, which of course crashed after a while etc etc. Would be great to see what an alternative would look like.

#3

Thanks. Feel free to sign-up on the site, although without the plug-in you won’t be able to get any data in there!

#4

@jouked it would be interesting if you could post some of your favourite images from influx and Grafana with some explanation of what it is (and why it’s useful) and I’ll see if I can replicate something similar.

#5

I’m happy to say that the ConstantGraph logging and graphing plug-in has now been approved in the app store here: MiOS Apps. To be fair to Ezlo I had made an error in the config and had to resubmit for approval. I’ve also noticed that the app is still showing a couple of errors during start-up. However, if you navigate away and come back 30 seconds later, it seemed to work - I’ll fix these in the next version.

I’ve been amazed at the response already with people signing up to the site so far. There have also been some ideas for new features which I am now working on but please keep the ideas coming as I want to make it as useful as possible.

Try it out and let us know what you think.

#6

Just released a new gauges feature so you can now show the current value of a channel on a gauge

Screenshot_20210311-020524
Screenshot_20210311-0205241281×2048 180 KB

#7

Hi - help required!

The app crashed at some point on my box last night and I had to reboot to get it going again. Unfortunately my log file was rotated so I can’t see what caused it. Did anyone else have the same problem? I want to see if it was the plug-in that caused the problem or was it a more general Vera issue? If you go onto the ConstantGraph tab of the plug-in and click on the version number at the bottom of the page - it’s a link to a debug page - if you had any issue could you post the contents here?

The link looks like this:
http://[vera ip address]/port_3480/data_request?id=lr_dsDebug_[device number]

Thanks!

#8

I installed the plugin on my vera a couple of days ago. For whatever reason, the plugin status is stuck at ‘initialising’.

What’s the best way to troubleshoot this?

#9

Hi. Thanks for giving it a go. I suspect there will be a few issues to sort out on different set ups. If you go onto the screen called ConstantGraph and click on the link at the bottom with the version number in you will go to a debug page. Can you post the contents of that page and I’ll see what the problem is. Thanks!

#10

Not much in there :thinking::

-1-===========================================================================
50 03/15/21 19:47:27.178 luup_log:31: Device Number: 300 v.device_type: urn:constantsphere-com:device:DataSync:1 v.device_num_parent: 0 v.id: <0x2ce1d680>
50 03/15/21 19:47:27.390 luup_log:31: Device Number: 300 v.device_type: urn:constantsphere-com:device:DataSync:1 v.device_num_parent: 0 v.id: <0x2ce1d680>
-2-===========================================================================
deviceId:300
apiKey length:16
dataMineDirectory:@nil
syncWithDataMine:0
Status:initialising
loggedIds:[{“D”:31,“E”:1,“I”:1,“S”:4},{“D”:196,“E”:1,“I”:2,“S”:4},{“D”:196,“E”:1,“I”:3,“S”:3}]
serviceList:[{“Id”:1,“Service”:“urn:micasaverde-com:serviceId:DoorLock1”,“Variable”:“Status”},{“Id”:2,“Service”:“urn:micasaverde-com:serviceId:EnergyMetering1”,“Variable”:“CurrentTemperature”},{“Id”:3,“Service”:“urn:micasaverde-com:serviceId:EnergyMetering1”,“Variable”:“KWH”},{“Id”:4,“Service”:“urn:micasaverde-com:serviceId:EnergyMetering1”,“Variable”:“Watts”},{“Id”:5,“Service”:“urn:micasaverde-com:serviceId:HVAC_OperatingState1”,“Variable”:“ModeState”},{“Id”:6,“Service”:“urn:micasaverde-com:serviceId:HaDevice1”,“Variable”:“BatteryLevel”},{“Id”:7,“Service”:“urn:micasaverde-com:serviceId:HumiditySensor1”,“Variable”:“CurrentLevel”},{“Id”:8,“Service”:“urn:micasaverde-com:serviceId:SecuritySensor1”,“Variable”:“Tripped”},{“Id”:9,“Service”:“urn:upnp-org:serviceId:Dimming1”,“Variable”:“LoadLevelStatus”},{“Id”:10,“Service”:“urn:upnp-org:serviceId:SwitchPower1”,“Variable”:“Status”},{“Id”:11,“Service”:“urn:upnp-org:serviceId:TemperatureSensor1”,“Variable”:“CurrentTemperature”}]
nextId:4
updatePeriod:30
enabled:1
logLevel:2
version:0.870
-=============================================================================

Anything in there that point to the issue you think?

#11

I can’t really see much from that. Would you mind restarting your Vera box give it a few minutes and then send the same thing again? It’s probably something during startup and your log file has probably already rotated since then.

#12

Yeah, there is some better info in there:

-1-===========================================================================
09 03/15/21 22:11:14.082 JobHandler_LuaUPnP::Run device 300 ptr 0xafcd70 ConstantGraph DataSync room 0 type urn:constantsphere-com:device:DataSync:1 cat 0:-1 id parent 0/0xbcec58 upnp: 0 plugin:9322 pnp:0 mac: ip: <0x2b2be310>
50 03/15/21 22:11:34.115 luup_log:300: DataSync: [2]Initialising dataSync System (0.870) <0x2bd01680>
50 03/15/21 22:11:39.107 luup_log:300: DataSync: [2]decoding manual service list <0x2ef01680>
50 03/15/21 22:11:39.290 luup_log:300: DataSync: [2]response: <0x2ef01680>
01 03/15/21 22:11:39.292 e[31;1mLuaInterface::CallFunction_Timer-5 function transferManual failed [string “–DataSync: for ConstantGraph…”]:344: attempt to index global ‘response’ (a nil value)e[0m <0x2ef01680>
50 03/15/21 22:12:23.138 luup_log:31: Device Number: 300 v.device_type: urn:constantsphere-com:device:DataSync:1 v.device_num_parent: 0 v.id: <0x2d301680>
50 03/15/21 22:12:23.235 luup_log:31: Device Number: 300 v.device_type: urn:constantsphere-com:device:DataSync:1 v.device_num_parent: 0 v.id: <0x2d301680>
-2-===========================================================================

#13

Hi @jouked. It looks like there is some error in the command being sent to the server but I can’t tell what - possibly a special character in the name of one of your devices? Could you go to Control > Advanced > Variables and set logLevel to 3 then restart your Vera and email me the output from the debug to contact@constantgraph.com. At log level 3 your APIkey will be in the log so don’t post it on a public forum. Thanks! Also make sure you change the log level back to 2 afterwards otherwise your log file could get filled up pretty quickly.

#14

I’ve managed to replicate the same error by putting an apostrophe in a logged device name. If that is the issue you are having the attached file should fix it. Unzip it and upload the .lua file to your Vera box via Apps > Develop Apps > Luup files > Upload (keep Restart Luup after upload checked). If it works for you, I’ll get it uploaded to the app store. Thanks for finding a bug for me!
L_CSDataSync.zip (5.6 KB)

#15

I’ve just sent a level 3 logfile to you via e-mail. I think my issue is not related to special characters, but we’ll see!

#16

hi @jouked. I think you are correct! The request looked valid but it didn’t get sent. Are you able to log onto your box and execute the following command and tell me what the response is:

curl -H ‘Content-Type: application/json’ https://webhooks.mongodb-realm.com/api/client/v2.0/app/constantgraph-iwfeg/service/http/incoming_webhook/data?apiKey=XXXXXXXXXXXXXXXX -d ‘{“app”:“vera”,“version”:0.880,“channels”:[{“id”:1,“v”:“1710”,“Name”:“Energiemeter - Watts”,“Device”:31},{“id”:2,“v”:“0.9”,“Name”:“Wasmachine - Watts”,“Device”:196},{“id”:3,“v”:“589.9437”,“Name”:“Wasmachine - KWH”,“Device”:196}]}’

But replace the X’s with your API key. Could there be some firewall stopping the request from getting out? Also, could you tell me what sort of Vera box you are running on.

#17

Well, looks like you’re on the right track. The request throws this error:

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: curl - SSL CA Certificates

I’m running a Vera 3 with the last available firmware for that platform. Guess I’m missing some modern root CA’s on my box.

#18

I did a quick try with the same request, using the -k option (insecure). This gives the following response:

{“error”:“{"message":"invalid character ‘â’ looking for beginning of value"}”,“error_code”:“FunctionExecutionError”,“link”:“App Services”}

Not what I expected. I’ll try to get the right certificates on my vera, see what happens.

Edit: The Vera 3 uses LuaSec 0.4, with default settings not to verify any certificates. So probably the certificate error is specific to curl, and the json error message is the actual error.

#19

@jouked - so I dug out an old Vera box that I had and tried the curl command on that and got the same SSL error you got. When I tried with the -k option it worked. I can see the invalid character error you got above in my server logs so that’s a definite step forward. I’m not sure why you would get that error if that character isn’t in the request! I’ll try updating the app to give the option of including the -k switch. Thanks for your investigations!

#20

@jouked - I have a new version of the code which includes an option to disable certificate checks (i.e. it adds the -k option to curl). It may not help you directly because of the invalid character issue you were also getting, but it should help some people (it works on my old Vera box). Unzip it and upload the .lua and .js file to your Vera box via Apps > Develop Apps > Luup files > Upload (keep Restart Luup after upload checked). Now that we know what the issue is, you can reset your log level to 2 (level 3 might use up a lot of space) DataSync.zip (8.8 KB)