guessed, very cool and super helpful. Thanks very much!!
PS: It was after reading Taken’s “book” on ISY / GEM integration that prompted me to look at Vera for energy reporting 
Hi again
I apologize in advance for my inexperience. I turned on my Vera today for the first time, added the Nest plugin, got the Sonos plug-in running, got the ISY manual loading plugin running (needed to learn about WinSCP for that) and am now trying to climb the hill described in this thread…
In Vera I added a smart plug and Nest T-stats (via the nest plugin) and have created a “Site” and a couple of “Devices” at SEG and see how to get the token numbers. BUT: I am not sure how/where to write the script guessed provided on the prior page. I’m not a programmer and don’t know Vera, so my questions are VERY basic…
[ol][li]Where do I write your script (with the edits relating to my SEG/Vera devices)[/li]
[li]Are all the parts of your script used for for a single device,e.g. a smart plug, or just part of the script (which part)[/li]
[li]Would you mind giving a basic step-by-step for setting up a single smart plug, for example, to report to SEG? I will then use that to expand on my own as much as possible[/li][/ol]
THANKS YOU VERY MUCH AND SORRY TO BE SUCH A NEWBIE
[ul][li]1) The script belongs in the Startup Lua section of Vera.
In UI5, this is under:
Apps → Develop Apps → Edit Startup Lua
Don’t forget to confirm the changes, and Save/Reload Vera to force this to startup.
[/li]
[li]2) The Script just does a pick-n-pull of any set of “State Variables” you want to send to SEG.
This can include any combination of things really, from any Vera Device, but typically it’s Power, Energy & Temperature related values.
You need only know the correct ServiceId/StateVariable, and Device#, that you want to Monitor. It’s a list, so it can be as many of these as you want. The code monitors for changes in the value, and just sends it over to SEG.
Note that SEG may not want you sending 100’s of values/second, and it will take your “SEG Device” offline if you sent at a high rate. If you’re using the Brultech Power Monitor Plugin, set the GEM to refresh data every 60-90s seconds (the default is 5s, mine is 90s)
[/li]
[li]3) Step by Step…
Not totally step-by-step, but the basic process is:
a) Modify the script to include YOUR SiteId, as well as the list of DeviceId/ServiceId/StateVariable combinations
Save this, and Vera will start pushing data to SEG, on your SiteID.
b) Login to SEG in a browser, and navigate to:
My Things > Tools > Discoveries
and set it running to discover your stuff. After a few minutes, it’ll see your feed data and present you with a UI to include it.
At this point, the Feeds will become a Device/Stream combination. You’ll have a Stream, within the [SEG] Device, for each thing you setup in the Vera/SEG startup script.
If you’re not seeing anything here, you can also try:
My Things > Tools > API Trace
and this will let you see what Vera is sending over. After a few minutes, if you still see nothing, then either Vera isn’t sending stuff OR, more likely, you’re using the wrong Token for SiteID.
c) Edit the Streams to add any metadata that you want.
This might include better Title/Description(s), along with Units-of-measure (in cases where SEG wasn’t able to introspect it from the Stream)
At this point you’ll have basic graphs and the data will start accumulating. From there, once you have some data, it’s probably a good point to build a simple one-sensor dashboard.
[/li][/ul]
…wonderful. Thanks guessed!!
do you guys know it the script should work in UI7 on vera edge? i copied/pasted it in, edited per instructions , but don’t see the connection at SEG. possibly i made an error and will post screenshots, but first figured I’d inquire if it should work with UI7
thanks!
It’s a fairly simple script, so there’s no reason it shouldn’t work on all Lua-enabled MiOS engines. That said, I’ve not tested this specific item on UI7, so it can always be a dice-roll.
Double check what you C&P’d, as well as any errors reported in /var/log/cmh/LuaUPnP.log since it’ll tend to show up there if there’s a typo.
Also, when looking at SEG, there’s a “Trace” mode in their User-toolbox that can be used to see what is being pushed to SEG. The most common problem, if you’re not seeing data there, is using an incorrect token in the push.
Hi Guessed
Two things:
#1 I want you to know I linked to your script on the ISY forums in hopes it would help us figure out how to transmit from ISY to SEG. I hope that is ok with you. Here is the post:
http://forum.universal-devices.com/topic/12682-energy-monitoring-circuits-plugs-interface/?p=132278
#2 My Vera was having issues the other day. That is why I did not connect to SEG. Your script works fine in UI7. I will separately post a few observations/questions. Once I better understand those, I will try to write another post in this thread spelling out a newbie step-by-step for this from the info you and others have provided above (to help consolidate the info in these 6 pages for other beginners, like myself)…
Thanks!
Yeah, it’s public content/posting here, so feel free to point to it.
Feel free also to create a break-out thread that just handles SEG, as needed. I buried it here because it’s the thread I’ve been using for all updates on my energy monitoring, and curbing energy usage, discussions.
As long as it remains a free-to-use option, with open source-code disclosure, I’m ok with others using it anyway they want
Either is fine with me. Would you prefer the SEG linking info be in a separate thread. If you don’t, I’d stay here.
Probably easier for users to find if it’s a new Thread. Feel free to start a new one, and you can just add a link back to here for reference (and link a post here, to the new thread also, so people can stumble across either)
That way, folks just interested in SEG for Data Capture, they’ll have an easier experience
That sounds perfect… I’m banging my head against a brick wall on this, but will withhold comments until the new thread is started.
That sounds perfect… I’m banging my head against a brick wall on this, but will withhold comments until the new thread is started.[/quote]
I was too but then got it. Once I did I saw that Guessed’s instructions were perfect. I had to pick from a few posts to pull it together and again once I did I saw the simplicity of what he created. I’m on my way up to the ski mountains at the moment so it might be Sunday or beginning of the weekfor a new thread. In the meantime, I’m sure your basic questions would be answered here
sorry I was delayed in doing this this week. I will try to work on it today or this weekend?
I forked it out this morning. Feel free to use extend the discussion here.
As a side-note, I also made an openHAB version of the script, for those that are looking at that route.
Vera SEG Step-by-Step Guide
The following is an (overly detailed) beginners step-by-step guide for linking Vera devices to SEG. 100% of this comes from @Guessed?s script and his instructions in this thread. I am doing this because as a beginner it took me a while to understand what he succinctly wrote. In case others are like me, this may be helpful:
INTRODUCTION:
Purpose: Monitor energy use and other stream data over time on a great, flexible and easy to use website accessible from all devices. SEG (Smart Energy Groups) is such a site. Thanks to @Guesseds script we can easily send all stream data from Vera to SEG. Stream date includes things like (i) energy data from smart plugs, home circuit panel or individual circuits, (ii) environmental conditions like indoor/outdoor temp, wind, humidity (iii) pretty much anything else that changes over time.
Examples:
Below are examples of the energy monitoring products.
In my case I have about 10 smart plugs connected to AV and PC equipment, sump pump and other high energy users I want to monitor at the plug level. I also have the Aeon HEM on my panel and have a GEM on order to grab all circuits. I also pick up indoor temperature and humidity from my NEST t-stats and outdoor temp, wind, etc. from the Underground Weather plug-in.
Context: Going into this I was using 1,600 KWH/month 20,000 KWH/year and DO NOT have electric heat or use AC. Thats a huge #!! By using a kill-a-watt meter and smart plugs to map every device?s on/standby electric draw and then Vera (and an ISY) to control everything I have cut this almost in half over the past month and have more to do.
INSTRUCTIONS:
On Vera
- Copy into a text file to edit @Guesseds script in post #5, page 1 of this thread, http://forum.micasaverde.com/index.php/topic,31212.msg175678.html#msg175678
In your pasted text file, edit only this highlighted portion of the text, as follows:
SEE ATTACHED IMAGE FOR EXACT LINES TO EDIT
-
Where is says ", remove the < and > symbols and put your own SEG Site ID token between the quotation marks. (In the SEG instructions below I?ll describe how to set up SEG and get that token, or if you want see/do SEG step #1, below and then continue with the Vera Steps.)
-
The list after the words SEG_DEVICES = { are to be edited as follows:
- The concept is you will have one line for each Vera data stream you want to send to SEG. The example in the script includes three streams (3 lines). Note: a single Vera device could have more than one stream and thus more than one lines, e.g. Nest has temp and humidity. Smart plugs have WATTS and KWH. You decide what to report and make a line for each. Each line will create a stream item at SEG (we?ll get to SEG in a moment).
NOTE: That you are about to do is fully editable in Vera. Therefore you may want to start with 1 or 2 lines (for 1 or 2 streams), as described below and then add additional lines once it is working for you.
3(a) Using the 1st line of the script SEG Devices table as the example, where the text says: stream=‘t_upstairs’, you replace the text inside the single quotes with whatever you want to SEG to identify that stream by. This does not have to be the name of the stream on SEG, you can change the SEG stream name, but not the SEG stream ID. You are creating the stream ID here. I don?t know if SEG likes spaces, caps, etc, so dont use them.
- For energy and temp streams (and maybe others, but I dont know) a prefix in the stream ID tells SEG to use WATTS, KWH or Degrees. You can edit that at SEG so don?t need to use prefix. Prefixes are e_ for energy (Watts), p_ for power (Watt Hours) and t_ for temp (Degrees).
3(b) Where the text says deviceId=335 you put the Vera device #. THIS IS NOT the ZWAVE NODE #. In UI7 the Device # is seen at the top of the Advanced tab on the device (just below the green header)
3(c) I am skipping to the end of the text line where it says serviceVar=“CurrentTemperature”}, (Ill come back to the middle part below). You will replace the text between the double quotes as follows: On Vera go to the Advanced tab for the device. Go to the Variables column. Find the variable you want to stream, e.g. WATTS, KWH, CurrentTemperature, CurrentLevel, WindSpeed, or whatever variable you want that Vera lists for that device. Copy/paste the variable field label name.
3(d) Now moving backwards in the line from the script text, where it says serviceId=‘urn:upnp-org:serviceId:TemperatureSensor1’ replace the text between the single quotes as follows: On Vera hover your mouse over the service variable label you used in step 4, above. When you hover you will see the service ID text to use.
Examples are:
urn:upnp-org:serviceId:TemperatureSensor1,
urn:upnp-micasaverde-com:HumiditySensor1,
urn:micasaverde-com:serviceId:Weather1.
NOTE on steps 3(c) & 3(d): I have NOT yet properly configured my Vera/SEG for the Weather Underground Plugin streams and possibly not all service variables and corresponding service IDs are supported by the script. More likely I made a typo…TBD
Here is a link to the Vera Wiki discussing Service Variables and Service ID: http://wiki.micasaverde.com/index.php/Luup_Variables#Device_category_3
3(e) On Vera, the final step is to paste your edited script into the Startup Lua, as follows: (i) copy the text file you just edited. On Vera navigate to Apps ->Develop Apps → Edit Startup Lua. Paste the text into the box. Click Go. Enter ctl-F5 to restart the Vera engine, now incorporating this script. You are now done on the Vera side.
On SEG
- Open a free account here: https://smartenergygroups.com/ . Then click My Things → My Sites and create a site. At the bottom of the new site info box you it says API Details and below that Site Token. The number below is what you want. This is the number you put in Vera Step #2 above where the script says says ?", (As stated above remove the < and > symbols and put your own SEG Site Token between the quotation marks).
If you previously created a SEG Site find the token by clicking on your site thumbnail image and then clicking the pencil and paper icon (the Edit icon) Under API Detail is your Site Token. (Be sure you are getting the Site Token, not a Device or Stream token).
-
On SEG create a new Device. Name it anything, but be sure to make the Node Name vera (or you can change the node name, but also edit the Vera script changing {node=‘vera’, to the name you use on SEG). Make Device Type be Display.
-
Now you are ready for SEG to receive the streams from Vera. To do this on SEG click Tools on the right of every screen, then Discoveries → Discover. Wait a few minutes and let it run. It is looking for the stream data Vera will send.
NOTE: Per @Guessed, Vera only sends data when it changes (or SEG only logs it when it changes). Either way, I recommend on Vera select Poll Now for the device(s) you are sending to Seg. You may then want to change the data on Vera, e.g. turn smart switch on/off and Poll Now each time to have changing data for SEG. I don?t know if you need to keep Discover running on SEG until it picks up all your streams, or just the 1st. In any event you can repeat Discover. To keep Discover running I log into SEG on another browser tab and click on my vera Device in SEG to see if any streams have been created (while Discover continues in the other browser tab).
- Once 1 or more streams show up in SEG, you can click see more detail and time periods and also to edit them. Mine usually default to Kilo eg KW and KWH (vs W and WH). To get to W WH edit the stream and select None if Kilo is selected. Also, on the edit screen set the Type of Data to correspond to the Vera Service Variable you picked, e.g. WATTS.
Just remember, both SEG and the Vera script can be changed if things aren’t perfect.
On SEG there are lots of things you can do with the data as @Guessed describes and shows in screenshots earlier in this thread.
That’s all for now
Have fun!
Thanks @Guessed!!!
-
Why did you choose display and not energy meter on the SEG site.
-
I would recommend that for energy devices on SEG one defines both the energy and power for a device as such my Aeon SmartStrip looks like this
SEG_DEVICES = {
{node='vera', stream='p_AeonSmartStrip', deviceId=115, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="Watts"},
{node='vera', stream='e_AeonSmartStrip', deviceId=115, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="KWH"}
}
- Whats the point in only sending energy changes, doesn’t it need to send a reading for every device on a poll frequency? Is there enough data fidelity if it is once in a blue moon or does it assume that the rate holds even over the time? My data is steady state, as such i have not had a reading show up and i can’t turn the strip off as my PC, Vera and other essential devices on it! How can i boot strap a reading (both channels showed up). Also i notice that the energy is chaging eg. 8.1 watt then 8.4 watts then 8.7 watts in the ISY. On the Vera it rounds up to 8 in the UI - does this mean the script wont send anything because it didn’t detect a change (PS turning off the strip made no difference either).
At this point i have the 2 streams appearing in SEG, but no data. It also shows only as being online in SEG 15 minutes ago…
I moved to pushing Energy data, directly, using BTMon.py to improve Vera stability. It’s very reliable, so I only needed to push infrequently changing aux-data from Vera.
I’ve recently stopped pushing any data from Vera, and now push all of my SEG data directly from openHAB.
2) I would recommend that for energy devices on SEG one defines both the energy and power for a device as such my Aeon SmartStrip looks like this [code]SEG_DEVICES = { {node='vera', stream='p_AeonSmartStrip', deviceId=115, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="Watts"}, {node='vera', stream='e_AeonSmartStrip', deviceId=115, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="KWH"} }[/code]
Yup, different (but related) quantities. Both are quite useful, depending upon what you’re trying to analyze/fix.
3) Whats the point in only sending energy changes, doesn't it need to send a reading for every device on a poll frequency? Is there enough data fidelity if it is once in a blue moon or does it assume that the rate holds even over the time? My data is steady state, as such i have not had a reading show up and i can't turn the strip off as my PC, Vera and other essential devices on it! How can i boot strap a reading (both channels showed up). Also i notice that the energy *is* chaging eg. 8.1 watt then 8.4 watts then 8.7 watts in the ISY.
Rev 1. That script is fairly simple, and only pushes data-changes. The openHAB one pushes on change, but also every 2 minutes. This can help with some graphing tools, as some don’t handle data streams of vastly differing period.
The script itself is agnostic to what’s generating the data, and on what frequency. Always room for tuning…
On the Vera it rounds up to 8 in the UI - does this mean the script wont send anything because it didn't detect a change (PS turning off the strip made no difference either).At this point i have the 2 streams appearing in SEG, but no data. It also shows only as being online in SEG 15 minutes ago…
At some point, tiny values for Wh and/or W are going to be in the level of error (typically 1-3% for a GEM, depending upon the CT used) so I wouldn’t sweat that resolution too much.
… esp when your Daily consumption is going to be measured, and billed, in KWh, not Wh 8)
For debug, use the SEG “Tools” menu, and enable Trace for a bit. It’ll give you more information about how it’s seeing the data stream you’re sending.
It’s very easy to get the Stream-name incorrect, and not have it pickup the stream.
[quote=“guessed, post:37, topic:186392”]
3) Whats the point in only sending energy changes, doesn’t it need to send a reading for every device on a poll frequency? Is there enough data fidelity if it is once in a blue moon or does it assume that the rate holds even over the time? My data is steady state, as such i have not had a reading show up and i can’t turn the strip off as my PC, Vera and other essential devices on it! How can i boot strap a reading (both channels showed up). Also i notice that the energy is chaging eg. 8.1 watt then 8.4 watts then 8.7 watts in the ISY.
Rev 1. That script is fairly simple, and only pushes data-changes. The openHAB one pushes on change, but also every 2 minutes. This can help with some graphing tools, as some don’t handle data streams of vastly differing period.
The script itself is agnostic to what’s generating the data, and on what frequency. Always room for tuning… ;-)[/quote]
Guessed
I wish I was not such an idiot and knew how to do basic scripting ???
Would providing the line(s) to tell Vera to poll and send current reading to SEG every X seconds/min, be a modest task to request from you? It would be a huge help for me/us and I/we would be extremely appreciative!!
This would greatly add to the functionality. I had primarily looked at RAW data at SEG. When I look at 30 min data, I see there are huge gaps in the streams when devices are off, asleep or constant. In my HA energy conservation endeavors, I hope to maximize those periods. I didn’t realize this until mentioned above.
Fingers crossed…
Thanks
@ccclapp ,
I’m not actively developing new code for Vera. My time is spent with openHAB which, for me, is a better fit for whole-house automation/measurement/tuning needs.
I’m sure someone who still uses Vera will chime in with aid.
Guessed, that’s too bad, but I totally understand. Are you willing to give the next guy a brief description of the conceptual steps required for that code update? I would imagine it would be more feasible for someone to try to pick up where you left if they had a simple roadmap (or at least some clues), as you likely have the dual knowledge of SEG and Vera better than anyone else.
Thanks you very much for writing your existing script!! It is the reason I bought a Vera (as plug reporting to SEG is not possible directly from an ISY).