Energy/Temperature Publication using SmartEnergyGroups.com (SEG)

For someone that knows Lua, it’s not that complex.

It’s basically:

a) Add a new, parameterless, function that can be used with [tt]luup.call_timer()[/tt]
Put a loop in there that goes over all the values in the Array, and builds the complete “message” to send to SEG. This will be similar to the existing one, except it’ll build the entire list, not just the piece that changed.
This will use a for loop

The guts of this function will be of the form (syntax & output not checked):

[code] local segData = “”
for k, v in pairs(SEG_DEVICES) do
segData = segData … string.format(“(node %s ? (%s %s))”, v.node, v.stream, luup.variable_get(v.deviceId, v.serviceId, v.serviceVar))
end

segData = string.format("(site %s %s)", SEG_SITE, segData)
result, status = http.request(SEG_URL, segData)

[/code]
b) Set this new function to run under [tt]luup.call_timer()[/tt] every “nn” seconds.

Thanks Guessed. Hopefully there is an Heir Apparent in the wings…

[quote=“guessed, post:39, topic:186392”]@ccclapp ,
I’m not actively developing new code for Vera. My time is spent with openHAB which, for me, is a better fit for whole-house automation/measurement/tuning needs.

I’m sure someone who still uses Vera will chime in with aid.[/quote]

Understand, I am trying to convince Ben at Brultech to get their DashBox to pull the data from ISY over IP REST interface - not sure I am getting far there…

[quote=“scyto, post:43, topic:186392”][quote=“guessed, post:39, topic:186392”]@ccclapp ,
I’m not actively developing new code for Vera. My time is spent with openHAB which, for me, is a better fit for whole-house automation/measurement/tuning needs.

I’m sure someone who still uses Vera will chime in with aid.[/quote]

Understand, I am trying to convince Ben at Brultech to get their DashBox to pull the data from ISY over IP REST interface - not sure I am getting far there…[/quote]

I have a Dash on the way. As I understand it, with an ISY that will send to SEG fine. The missing link has always been for plugs (or low cost HON eg Aeon). I’ve got a lot of high users on 1-2 circuits, thus plug monitoring/reporting is critical, (at least for me and someone else without a GEM/Dash).

The dash will send the data collected by a GEM or ECM to SEG.
It can also send the data to the ISY.

I am not ware of anything that sends from ISY to SEG but ben just wrote a proto-script in his forum he things could do it fi there is data in ISY.

alex

I think you’re correct. GEM can nor report all channels directly to ISY but ISY doesn’t report ANYTHING to SEG.

Also, most relevant to this thread is the fact that on any device ONLY GUESSED’S SCRIPT HANDLES PLUGS. For me that’s a key missing link…
Any chance you can/would take a stab at adding the polling/frequency edit to his script as he described in response to your comment about the impact of the lack of it? I wish I could, but can’t script :frowning:

Me neither, I can read one and make educated guess, actually figuring out where the braces go and what should be between them - no chance :slight_smile: - unless I go pick up a book on lua scripting (my tip, download notepad++ on windows - makes reading these files a breeze)

I have now changed the script to send KWH data to SEG every 30 minutes. I have added a new parameter to each device I want to report to SEG, and modified the script to be used in a scene instead of under “startup-lua”. This way I can easyli adjust how often values should be send to SEG, and if a device is asleep for several hours and hence has not updated the KWH for a long time nothing will be send to SEG untill the KWH is updated by the device.

The reason I need the extra paramenter is because SEG needs the delta KWH hence I need to know the prevous send value in order to calucalte the delta.

I have created a new paramenter for each device with the same serviceId as the original KWH (under the advanced tab->new variable"

then created a new new scene running every 30 min doing nothing but running this LUA script ( you can of course keep the exisiting script in start up lua for watts devices that should report watts in “real time” to SEG)

local http = require(‘socket.http’)
http.TIMEOUT = 5

local SEG_SITE = “15e38bcc3f0d3d7”
local SEG_URL = “http://api.smartenergygroups.com/api_sites/stream
local CALC_SID=“urn:micasaverde-com:serviceId:EnergyMetering1”

SEG_DEVICES = {
{node=‘vera’, stream=‘e_beggekwh’, deviceId=3, serviceId=‘urn:micasaverde-com:serviceId:EnergyMetering1’, serviceVar=“KWH”},
{node=‘vera’, stream=‘e_northqt15’, deviceId=7, serviceId=‘urn:micasaverde-com:serviceId:EnergyMetering1’, serviceVar=“KWH”},
}

local i=1
while SEG_DEVICES[i] do
local v=SEG_DEVICES[i]
local oldCalc1 = luup.variable_get(CALC_SID, “KWHold”, v.deviceId )
local newCalc1 = luup.variable_get(CALC_SID, “KWHnew”, v.deviceId)
local oldCalc=tonumber(oldCalc1)
local newCalc=tonumber(newCalc1)
if (newCalc>oldCalc) then
luup.variable_set(CALC_SID, “KWHold”, newCalc, v.deviceId)
segData = string.format(“(site %s (node %s ? (%s %s)))”, SEG_SITE, v.node, v.stream, newCalc-oldCalc)
http.request(SEG_URL, segData)
end
i=i+1
end

I have only had the veralite for 1 month and this is my first script written i LUA, (and the first script I have programmed for the last 3 years), hence please bear with me if it could have been done smarter, but I had to reuse as much as possible from guessed original script :slight_smile:

Great stuff … I have setup 3 streams ( temperature , energy watts and Kwh ) and they are displayed correctly on the SEG site ( Thanks Guessed)

But i noticed that there are some disconnections of the streams as per alert received from the SEG site below .Is it a known issue ?

Recent Device Events

@ your device dsb05 is missing, last known value was: 21.0 C 7 hours ago

@ your device dsb05 is back! with value: 21.0 C 9 hours ago

@ your device dsb05 is missing, last known value was: 22.0 C 4 hours ago

@ your device dsb05 is back! with value: 22.0 C 17 hours ago

The two cases where I’ve seen SEG Report like that correlate to the client(s) “reporting too fast” and “reporting too slow”.

In my case, I had a few other things reporting across the same entity and/or there was a volatile value in the list of stuff I sent over to SEG.

With energy data, unless it’s some sort of daily-summary, I’d expect you’d get a reasonable level of volatility/change, so it’s possible that it’s too fast and SEG is throttling.

Quickest way to find out, esp if there’s a [relatively] constant pattern, is to look at the Tools/Trace output (that’s a SEG Menu/Tool) and see if your device is reporting (or not) during these periods.

For reference, my on-schedule Energy-based samples are sent to SEG ~5 minutes, with data collections every 90s. My on-change openHAB transmissions are based upon change of the source data (in my case Nest-openHAB, polling every minute, and Weather, polling every 30 minutes)

OK @Guessed,

The decision to have on or off status of a stream is based on the missing period parameter in the API details ( 5000 Seconds )

As per the API trace , it is confirmed that the missing requests from Vera are the cause of these events due to no change reported to SEG within the API missing period of 5000 Seconds :

  • The wall plug is always making energy report to SEG within the 5000 s period of time
  • The 4in1sensor (DSB) does not report temperature report sometimes to SEG within the 5000 s period of time.

I was able to force a temperature report by changing the location of the DSB05 multisensor and to have then the status back on line for this device.

Conclusion : The root cause is sometimes too slow data , but this is consistent with the behavior of the devices which sends report only in case of change of value ( temperature or watts/kwh ).

Some posts back, I pushed what was an almost-complete mod to the existing code to make it push data periodically.

It only needed to be wrapped in a function declaration, and scheduled/re-scheduled via luup.call_timer() and it would provide both “periodic” as well as the existing “change-only” push to SEG.

That’s one option.

The next is to try the Scene-implemented version posted above, with the Scene acting as the timer.

The third it to include “something” in the SEG list that does change often enough… Outdoor Temp, or anything really… it doesn’t have to be data that SEG actually graphs :wink:

[quote=“celbrandt, post:48, topic:186392”]I have now changed the script to send KWH data to SEG every 30 minutes. I have added a new parameter to each device I want to report to SEG, and modified the script to be used in a scene instead of under “startup-lua”. This way I can easyli adjust how often values should be send to SEG, and if a device is asleep for several hours and hence has not updated the KWH for a long time nothing will be send to SEG untill the KWH is updated by the device.

The reason I need the extra paramenter is because SEG needs the delta KWH hence I need to know the prevous send value in order to calucalte the delta.

I have created a new paramenter for each device with the same serviceId as the original KWH (under the advanced tab->new variable"

then created a new new scene running every 30 min doing nothing but running this LUA script ( you can of course keep the exisiting script in start up lua for watts devices that should report watts in “real time” to SEG)[/quote]

Hi celbrandt

I’m traveling for a bit and can’t try your script, but wanted to thank you for your effort in adding periodic poling/reporting to guessed’ script.

Is it working well?

Thanks again I look forward to this!

Hi CCCLAP

my script seems to Work flawlessly. The last three days, KwH consumption for the last hours has been reported for 3 different stream every hour without any glitches.

But we still owe it all to Guessed. Without any LUA experience I could not have done it without all the hard work done initially by Guessed !

hoping someone can help me troubleshoot this. I initially set up the original script to send energy and power data from an aeon labs smart switch hooked up to my washing machine. In SEG I created the dummy device, did the discovery, and it discovered the two streams and connected them to my device. Great so far.
I then tried to add streams for 2 home energy monitors. 2 monitors - power and energy for each. I’m not having any luck picking them up in SEG. When I do an api trace I almost never see anything at all in there, but the 2 streams for my washing machine device do regularly update. I tried to manually create another device thinking maybe I need to do that first, but I get a message about ‘permalink already used’.

Here’s my script

local http = require('socket.http')
http.TIMEOUT = 5
 
local SEG_SITE = "478c91a17fca017"
local SEG_URL = "http://api.smartenergygroups.com/api_sites/stream"
local segLog = function (text) luup.log('SEG Logger: ' .. (text or "empty")) end
 
SEG_DEVICES = {
    {node='vera', stream='p_washing_machine',   deviceId=19, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="Watts"},
    {node='vera', stream='e_washing_machine', deviceId=19, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="KWH"},
    {node='vera', stream='p_hem1', deviceId=28, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="Watts"},
    {node='vera', stream='e_hem1', deviceId=28, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="KWH"},
    {node='vera', stream='p_hem2', deviceId=37, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="Watts"},
    {node='vera', stream='e_hem2', deviceId=37, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="KWH"}
  }
 
local function findStream(deviceId, serviceId, serviceVar)
  -- TODO: Change from a simple scan to something that'll support more entries efficiently.
  for k, v in pairs(SEG_DEVICES) do
    if (v.deviceId == deviceId and
        v.serviceId == serviceId and
        v.serviceVar == serviceVar) then
      return v.node, v.stream
    end
  end
 
  return nil, nil
end
 
local function initWatch()
  for k, v in pairs(SEG_DEVICES) do
    luup.variable_watch('segWatch', v.serviceId, v.serviceVar, v.deviceId)
  end
end
 
function segWatch(deviceId, serviceId, serviceVar, oldValue, newValue)
  local nodeName, streamName = findStream(deviceId, serviceId, serviceVar)
 
  if (nodeName == nil or streamName == nil) then
    segLog(string.format("Node not found for %s,%s,%s", deviceId, serviceId, serviceVar))
    return
  end
 
  -- TODO: Encoding
  segData = string.format("(site %s (node %s ? (%s %s)))", SEG_SITE, nodeName, streamName, newValue)
  segLog(segData)
 
  result, status = http.request(SEG_URL, segData)
end
 
initWatch()

I think I have it figured out now and I was just thinking of things on the SEG side in wrong way. I was expecting a device in SEG for each device in Vera. Now I set up 1 device in SEG called Vera w/ multiple streams coming from the different physical devices I have and everything seems to be working.

Is that the correct way to set it up on the SEG side?
Site = Vera
Device = Vera
Stream 1, 2, 3, 4, 5 etc. = p_hem1, e_hem1, p_hem2, e_hem2, p_washing_machine, etc.

Hum , I received a mail from SG this morning ,

[i]Smart Energy Groups has been free for the majority of our members now for a long time now, and we need your support in order to continue.

This means that Smart Energy Groups is transitioning to a paid service in order for it to continue.

When you log in next, you will be prompted to select from our Community or Premium plans. This will set you up with an account and all the other necessary things, like your credit card details etc.[/i]

EDIT : After checking the SEG site , the free subscription still exists but with only 3 streams