Contribution: Monitor and visualize energy usage with emoncms.org in Vera

Other Devices & Integrations Power Management
#1

I found a really nice looking service called emoncms.org to visualise energy consumption measured in Vera. It’s free and open source. You can use emoncms.org or setup your own server at home. You can also use it to log arbitrary numerical variables in vera, such as battery level, light, temperature etc. You can create custom dashboards and export to CSV.

Here is how I did it:

[size=18pt]Setup Vera (instructions for UI7)[/size]

[ul][li]Register for an account at emoncms.org[/li]
[li]Look up your write API key on your profile page[/li]
[li]Download the attached EnergyMonitor.lua[/li]
[li]Edit the file, enter your emoncms API key and setup your devices in VARIABLES according to the sample code already there[/li]
[li]Please note that the energy can be fetched in two ways. The first way is directly from the Watts variable of your device. The second way is to use a lua function to calculate the energy. The second option is useful if you don’t control the device directly from Vera. I use it to measure if my computer is on using a PingSensor, and send the power based on the sensor reading. I also have an ‘Other’ where I enter the average power of external devices such as Washmachines etc.[/li]
[li]In the Vera UI, go to Apps → Develop Apps → Luup Files[/li]
[li]On the right, click “Choose File”. Browse to and select your edited EnergyMonitor.lua file, and click “Upload”[/li]
[li]On the left, click “Edit Startup Lua” and add the following line:[/li][/ul]

require("EnergyMonitor.lua")

[ul][li]Click “Go”[/li]
[li]Enjoy your data flowing to emoncms.org![/li][/ul]

[size=18pt]Setup emoncms inputs and feeds[/size]
You will immediately find your data here: [url=http://emoncms.org/input/view]Emoncms - input view url
The inputs are by default not stored. Therefore, you will need to setup what is known as “feeds”. Feeds is where the data is stored.

Feeds and inputs are very flexible, and therefore not very intuitive in the beginning. There are some guides at the bottom of this page which explains how to create feeds from your inputs:
[url=http://openenergymonitor.org/emon/guide]Home | OpenEnergyMonitor

In short, what I did was this:

[ul][li]Go to your inputs at [url=http://emoncms.org/input/view]Emoncms - input view url
[li]Find the input called Total and click the Spanner icon to access its processes. [/li]
[li]You will now create three feeds:[/li]
[li]Enter the following under Add process: Log to feed / CREATE NEW: / Total W / Fixed Interval With Averaging (PHPFIWA) / 10 s. Click Add.[/li]
[li]Enter the following under Add process: Power to kWh / CREATE NEW: / Total kWh / Fixed Interval No Averaging (PHPFINA) / 10 s. Click Add.[/li]
[li]Enter the following under Add process: Power to kWh/d / CREATE NEW: / Total kWhd. Click Add.[/li]
[li]If everything is setup correctly, you will now find your feeds at [url=http://emoncms.org/feed/list]Emoncms - user login

[size=18pt]Setup emoncms energy app[/size]
Now for the fun part, setting up the dashboard.

[ul][li]Click Apps > My Electric[/li]
[li]Click the Spanner icon at the top right[/li]
[li]Choose feeds as follows:[/li]
[li]Power feed (Watts): Total W[/li]
[li]Bar graph feed: Total kWh[/li]
[li]Wh or kWh feed?: kWh elapsed[/li]
[li]Set currency and unit cost if you wish.[/li]
[li]Click Save.[/li]
[li]Enjoy your energy dashboard![/li][/ul]

[size=18pt]The result[/size]
Attached is the code and examples of the two dashboards. My Electric displays current and historic energy consumption. I played around with My Solar to visualize how much energy I could save if I would switch to LED instead of halogen lights.
You can also design your own dashboard. I recommend that you watch the YouTube introduction before designing.

This code could be made into a Vera app. If anyone feels like taking the time to make an app, feel free to use my code as a base - just mention my name somewhere :slight_smile:

Vera Plus - Issue to post energy values into emoncms
#2

Really realy nice, will give it a try! Good work!

#3

thanks for this…

still struggling to get the feeds running…

could do with some more examples…

track on time of a switch for example

#4

I found an error in the code and just uploaded a fixed file (deviceId was missing from VARIABLES).

I also made “Total” exclude any variables but Watts. That means that you can easily do arbitrary logging of switches, light level, battery level etc.
I included an example for a switch also:
Set up the feed as in my first post (first feed), that is, use the “Log to feed” processing. You can simply check on your feed by clicking on the “eye”-icon in the feed list. Attached is a sample of how my switch looks really zoomed in (zoom in by click and select in the graph). Not sure if that fits your needs. You can always export the feed data as CSV.

-- Setup your devices here. You can use a function to calculate the power as illustrated in the sample. -- For device logging, use: key, deviceId, serviceId, serviceVar -- For function based logging, use: key, calculate, serviceVar local VARIABLES = { { key="Livingroom", deviceId = 12, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="Watts" }, -- Send device energy { key="Kitchen", deviceId = 13, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="Watts" }, { key='KitchenSwitch', deviceId=13, serviceId="urn:upnp-org:serviceId:SwitchPower1", serviceVar="Status"}, -- Send switch status { key='Computer', calculate=function() return (IsComputerPingSensorTripped() and 38 or 1) end, serviceVar="Watts" }, -- Send variable value { key='Other', calculate=function() return 15 end, serviceVar="Watts" } -- Send a constant value }

#5

yes it looks quite good.
for your info, I made an native integration for it in ALTUI plugin. MiOS Apps

#6

quality…

Thanks for that

#7

@amg0, ALTUI looks like a great piece of work, I might try that sometime.
I looked through the documentation but I didn’t find any mention of emoncms. Did I miss something, or would you please shed a little light on the level of integration?

#8

[quote=“wezzix, post:7, topic:190754”]@amg0, ALTUI looks like a great piece of work, I might try that sometime.
I looked through the documentation but I didn’t find any mention of emoncms. Did I miss something, or would you please shed a little light on the level of integration?[/quote]yes it is integrated. From a variable you can configure a data push to emoncns.

#9

I am guessing it only pushes to one of thingspeak, economy or datayours?

#10

out of the box, it pushes any variable value to either thingspeak or DataYours2;
but it also provides an web interface to integrate some other data push destination. that requires though that a piece of code is calling a web service into ALTUI to register then it will receive, again as a web service, the value of the variables that the user has chosen to save; that piece of code can then do the push to whatever data storage they want.
more info here: http://forum.micasaverde.com/index.php/topic,35860.0.html

#11

UPDATE: Nevermind, figured out the device was urn:micasaverde-com:serviceId:EnergyMetering1

Having a bit of problem reading my energy monitor. The script seems to be communicating with Emoncms, but not reading the watts. Here’s the code I put in. I’ve also attached the device config. I’ve tried modifiying the key name to only use a number and not include any spaces, but it doesn’t seem to make a difference. Any ideas? Thanks!

local VARIABLES = {
{ key=“Grid Total”, deviceId = 25, serviceId=‘urn:schemas-micasaverde-com:device:PowerMeter:1’, serviceVar=“Watts” }, – Send device energy
{ key=‘Other’, calculate=function() return 15 end, serviceVar=“Watts” } – Send a constant value
}

#12

Awesome info Wezzix! I ended up writing a python script to pull SolarEdge data from their API and post to emoncms too. The code runs on a Synology NAS that I can detail if anyone wants more info. I have my HEM collecting usage data based on your script with the two being correlated on the My Solar app.

#! /usr/bin/python
import datetime
import http.client
import json, urllib.request

#Gather wattage from SolarEdge API
solarEdgeAPIKey=“APIKEY”
url = “https://monitoringapi.solaredge.com/site/1/overview.json?api_key=” + solarEdgeAPIKey
#url = “https://monitoringapi.solaredge.com/site/1/power?api_key=” + solarEdgeAPIKey + “&startTime=2016-02-16%2008:00:00&endTime=2016-02-16%2011:30:00”
response = urllib.request.urlopen(url).read().decode(‘utf8’)

data = json.loads(response)
watts = repr(data[‘overview’][‘currentPower’][‘power’])

#print(data[‘overview’][‘currentPower’][‘power’])

Local EMONCMS API Key

#EMONKEY = “APIKEY”

EMONCMS.org API Key

EMONKEY = “APIKEY”
now = datetime.datetime.now()

#—Local—#
#conn = http.client.HTTPConnection(“192.168.3.20:80”)
#request = “/emoncms/input/post.json?apikey=” + EMONKEY + “&node=1&json={power:” + watts + “}”

#—Hosted—#
conn = http.client.HTTPConnection(“emoncms.org:80”)
request = “/input/post.json?apikey=” + EMONKEY + “&node=20&json={power:” + watts + “}”

emoncms uses a GET not a POST

conn.request(“GET”, request)
response = conn.getresponse()

#13

Thanks mmd, I’m glad you found it useful!

#14

Hi mmd,

I’d love any more info you might have on pulling your SolarEdge data with the python script. I tried playing around with the script in your post, but kept getting urlopen error 3 for some reason :frowning:

Thanks!

#15

Hi All,

Had a go at setting this up today. Got the Lua file edited and uploaded and put the line into the Startup Lua to call it.

When I go to Emoncms - input view I’m not seeing any inputs, so I assume that something isn’t working as expected.

I’ve attached my lua file - is there anything wrong in the code or do I have some other issue?

UPDATE: If I paste the code into the Test Luup box, it works and starts sending updates to Emoncms. So for some reason the startup Lua code:

– emoncms.org data uploader
require(“EnergyMonitor.lua”)

Isn’t working - they are the first lines in my startup.

#16

You wouldn’t normally put the .lua extension into the require statement.

Does this work?

require "EnergyMonitor"
#17

Hi akbooer,

No joy with that code change either.

The only other stuff in my startup is related to logging and restarts, not sure if it has a bearing.

-- Keep snapshots of LuaUPnP.log 
local snapSecs = 180   	-- Seconds of log to keep in snapshot.
local snapNum = 5	-- Total number of snapshots to keep.
local baseFile = "/www/logsnap"
local snapFile = baseFile .. "1.txt"
-- Rename the snapshot stack
if snapNum > 1 then
	for i=snapNum-1,1,-1 do
		os.rename(baseFile .. tostring(i) .. ".txt", baseFile .. tostring(i 1) .. ".txt")
	end
end
-- Take snapshot of current log
local dt = {}
local cut = os.time() - snapSecs
local snapF,snapE = io.open(snapFile,"w ")
if snapF ~= nil then 
	local foundcut = false
	for line in io.lines("/tmp/log/cmh/LuaUPnP.log") do
		if not foundcut then
			mStr,dStr,yStr,hhStr,mmStr,ssStr = string.match(line,"^89.-(9 )/(, )/(e ) (t ):(e ):(o )")
			if mStr ~= nil then
				dt.month = tonumber(mStr)
				dt.year = tonumber(yStr)   2000
				dt.day = tonumber(dStr)
				dt.hour = tonumber(hhStr)
				dt.min = tonumber(mmStr)
				dt.sec = tonumber(ssStr)
				tstamp = os.time(dt)
				if (tstamp >= cut) then foundcut = true end
			end
		end
		if foundcut then
			snapF:write(line .. "\n")
			-- Following line stops snapshot at point of crash.
			if string.find(line,"LuaUPnP Terminated with Exit Code:") ~= nil then break end
		end
	end
	snapF:close()
end


-- Log each restart. 
local function restartLog(message, keepdays)
	local lfs = require "lfs"
	local socket = require("socket")
	local time = socket.gettime() or os.time()
	local tms = string.format(".d  ",math.floor (1000 * (time  1))) 
	local stamp = os.date("s o l  u",math.floor(time)) .. tms
	local baseFile = "/www/restarts"
	local logFile = baseFile .. ".txt"
	local lastLog = lfs.attributes(logFile,"modification") or time
	-- Log this restart	
	local uptime = time - lastLog
	local uptimeStr = string.format("Uptime: s Days 4 Hrs 9 Mins. ",math.floor(uptime / 86400),
			math.floor((uptime a86400) / 3600), math.floor((uptime o3600) / 60))
	local file = io.open(logFile, "a ")
	file:write(stamp .. "Restart. " .. uptimeStr .. (message or "").."\n")
	file:close()
	-- Remove old entries
	local prevDays = keepdays or 30   	-- Days to keep in addition to today.
	local tmpFile = baseFile .. ".tmp"
	local mTxt={Jan=1,Feb=2,Mar=3,Apr=4,May=5,Jun=6,Jul=7,Aug=8,Sep=9,Oct=10,Nov=11,Dec=12}
	local dt = {}
	local today = time - (time  86400)
	local cut = today - ( prevDays * 86400)
	local tmpF,tmpE = io.open(tmpFile,"w ")
	if tmpF ~= nil then 
		for line in io.lines(logFile) do
			dStr,mStr,yStr = string.match(line,"^(4 )3(8 )5(1 )9")
			dt.month = mTxt[mStr]
			dt.year = tonumber(yStr)
			dt.day =tonumber(dStr)
			date = os.time(dt)
			if (date >= cut) then tmpF:write(line .. "\n")	end
		end
		tmpF:close()
		local retn,err = os.rename(tmpFile, logFile)
	end
	lfs.touch(logFile,time)
end

local netOK = "OK"
if os.execute("ping -c 1 8.8.8.8") ~= 0 then netOK = "Offline" end
restartLog("Network: " .. netOK, 30)	-- Last argument is log history in days.
#18

So what does the log file show?

#19

Please review:
http://forum.micasaverde.com/index.php/topic,33226.msg244015.html#new

Add a line to the beginning of the LUA File:
module(“EnergyMonitor”, package.seeall)

For all of the functions that you callback by NAME from the various luup.xxx functions add a line as follows during some initialization (i.e. at bottom of file):

_G.FunctionName = FunctionName

i.e. EnergyMonitorOnTimer

#20

I feel like I’m bringing a knife to a gun fight talking code with Akbooer and RTS… :slight_smile: Thanks to both of you for taking the time to answer questions.

Ok, modified the EnergyMontor.lua (attached) to include the elements Richard describes above.

Replaced my Lua Start up with:

emoncode = require("EnergyMonitor")
emoncode.EnergyMonitorOnTimer()

The code runs. However, if I try to add my original startup code back (see previous snippets, it’s the log restart and capture luaupnp.log snapshots that’s posted on the forum), the EnergyMonitor code no longer runs and Vera gives me an “Error in Lua Scenes and Events” message. But if I leave the Energymonitor bits out of startup, the other code runs fine.