The Sad State of Energy Monitoring

I recently purchased and installed an Aeon Labs HEM. Based on what I’ve experienced & seen, here’s where it looks like we are (please correct me if I’m wrong):

  • The Ergy plugin shows data, but no options for disaggregation (it’s not very informative to just see that I’m using 1700 watts total).

  • The Ergy plugin may cause instability in Vera.

  • The paid Ergy plugin is not available until Vera finishes tweaking their software - UI7?

  • The Datamine plugin is no longer being actively developed, and also had no disaggregation capabilities?

  • The Datayours plugin is in beta, requires some level of advanced knowledge to install, and doesn’t offer disaggregation.

  • There is no plugin for Vera that offers disaggregation.

  • Bidgely offers disaggregation, but doesn’t interface with Vera, and doesn’t work with Aeon Labs.

Is there any ray of sunshine that I’m missing here? I wasn’t expecting a miracle, but certainly some higher level of software for use with the Aeon Labs HEM & Vera. I guess I thought energy monitoring would be more central to the home automation experience at this point.

If you’d like to describe a little more completely what you need, there might be someone willing and able to give it a go.

[tt]DataYours[/tt] is, indeed, still in beta, but the configuration challenges should go away with the first release. The only real questions which need to be answered for this are: “what data do you want?”, “where do you want to store the data?” and “how do you want to view it?”

I’m thinking about something like Bidgely, with an algorithm for usage disaggregation. This would be good for people who don’t want to spring for circuit or appliance-level monitoring and understand that they aren’t getting totally accurate numbers, but would like to see a breakdown instead of one big number.

This is certainly harder, it requires both quality monitoring hardware, frequent sampling, and robust algorithms derived from “big data”.

What you’re asking is energy monitoring without the hardware and, presumably, without the investment.

I’m not sure how Bidgely works, but it disaggregates the usage from the same kind of HEM’s as the Aeon. So hardware and data are not the issue I think.

In my imagination the way it works is for me to be able to tag specific draws and label them. I watch the usage when someone turns on the air conditioner, it’s a 2200 watt draw, I label that 2200 watt appliance as the HVAC. My oven is 4000 watts, the water heater is 4500, 13w draws are probably cfl’s, and so on. Doing it this way it wouldn’t be hard to isolate the main players and have a reasonably good picture.

As I say, I don’t know how Bidgely does it, but I think it’s something along these lines.

I see that Bidgely is careful to keep their algorithm a secret, which suggests to me that it’s complicated enough to have commercial value. It’s not something that I would particularly relish reimplementing. Especially not for the going price for Vera plugins.

If Bidgely wanted to open up its data submission interface for Vera to upload the energy data that it collects, then it’s possible that someone here would be tempted to write the glue code. Like ERGY, it would be able to collect data from any one of the existing energy monitors, so that users were not tied to one particular brand of energy monitor.

In the absence of cooperation from Bidgely, then someone would have to write their own analytics code. All the data is collected, and through plugins like datayours it can be logged offsite for processing. “Only” the disaggregation code is missing.

Oh, Bidgely does have a developer API. There is hope. (No, I am not volunteering to write the glue code.)

The script that I use for SmartEnergyGroups.com (SEG) could be adapted to push XML to Bidgely in their specific format.

See this post for the guts of the SEG data pushing script:
http://forum.micasaverde.com/index.php/topic,9998.msg175678.html#msg175678

The real question might be what you want to do with the data once it’s been disaggregated.

ie. Tuning, for cost-savings, or Automation (turn on something when the washer goes off)

If it’s Tuning, then the large-loads aren’t that interesting [IMHO] over the short-term… you can’t tune them, apart from simple Time-of-day/scheduling - stuff you can readily do without an energy monitor. Over the long term, their data might be more interesting though (to detect pending failure/need for repair)

For tuning, the smaller loads are often more interesting (lighting, vampire loads, etc) since there are more of them and they’re often the ones you’re less aware of… but these get completely lost in the noise and/or baseline usage in a house (at least they are in mine, until I look at the channel measures).

Given what I’ve seen, it would be impossible to separate out the smaller/tunable loads - or would trigger the purchase of a few more “one off” HEM’s to get finer-grained data, or more accurate measurement… at which point the multichannel devices are a better bet.

Thanks guessed. Pushing data to Bidgely would be one thing to consider, but I would like to keep everything on one platform. If I wanted to use Bidgely I could just get an HEM that worked with it.

What I am looking for is for tuning, or really, awareness and whatever comes out of it. We do building performance work so we are tuning the big loads, by tuning the house and systems. We’d like to be able to track the changes. Also, once you account for the large loads it shows you what’s leftover. Then you can then get to work with smart switches, or even just a kill-a-watt, to figure that stuff out.

I’m interested in this for myself, but I’m mostly looking for something to offer our clients.

I was pushing data to Vera but, through experimentation, it’s really not built for doing any [decent, dynamic] collection and analysis of the real volumes of data involved.

I was pushing ~24 channels of Energy data to Vera, every 5 seconds, and was looking to increase that with additional circuits, data analysis, and long-term data retention.

I also found that I wasn’t using this data for any type of automation, so having it pile-up in, and potentially further destabilize, Vera didn’t make sense (to me) - esp since it would require additional setup and maintenance of stuff on my side (Storage, Analysis tools, etc) that are replicating hosted ones that can do it better.

Outsourcing this data collection and reporting to SEG gave me mechanisms to:

[ul][li]a) do [manual] static, periodic, analysis of the loads[/li]
[li]b) measure the impact of my tuning[/li]
[li]c) customize visualizations/dashboards that I could present to folks in the house to give them an understanding of the things they can change.[/li]
[li]d) Login remotely and see my data while away.
I was just on a 2wk vacation, and was able to look at the baseline energy usage of the house (without using my VPN) using only an iPad. Discovered a bunch of new things to tweak in the system. :wink: [/li]
[li]e) Author custom monitoring for material deviations on circuit loads… in the future[/li][/ul]

Items (a) and (b) are something I wont expose to folks in the house, but they see it through the various purchases made (and the small PG&E Bill each month)

Item (c) is something I can embed in a Control-point as HTML, so it can visually co-exist with the controls of Vera, and look like it’s coming from a single spot, even when it’s not.

Also, once you account for the large loads it shows you what's leftover. Then you can then get to work with smart switches, or even just a kill-a-watt, to figure that stuff out.
Given the dynamic nature of what's "plugged in" to the smaller circuits, any static analysis done on these (through Kill-a-watt et-al) would likely be voided every 6 months or so. I'm constantly amazed at the "what's that" questions when I look at the data, esp when certain loads move around ;)

Anyhow, I wouldn’t worry about whether it’s integrated to Vera or not, at least at this stage. It really only needs to be there if you need the automation functions… or if you just want to use Vera as a programmable bridge (Energy Monitor on one side, and the particular Energy Analysis facility on the other)

It just seems too limiting to expect this platform to provide the capabilities of services like SEG, Bidgely (etc), which live in an energy-use focussed world.

Anyhow, as @futzle said, given the secret-sauce these companies have it might pay to push the data to them to see if it’s possible to deseg it in a meaningful manner. That type of experiment would give you a much better idea of whether it’s feasible to do what you’re looking to do (functionally) with a single HEM unit.

[quote=“fullmoonguru, post:1, topic:181387”]I recently purchased and installed an Aeon Labs HEM. Based on what I’ve experienced & seen, here’s where it looks like we are (please correct me if I’m wrong):

  • The Ergy plugin shows data, but no options for disaggregation (it’s not very informative to just see that I’m using 1700 watts total).

  • The Ergy plugin may cause instability in Vera.

  • The paid Ergy plugin is not available until Vera finishes tweaking their software - UI7?

  • The Datamine plugin is no longer being actively developed, and also had no disaggregation capabilities?

  • The Datayours plugin is in beta, requires some level of advanced knowledge to install, and doesn’t offer disaggregation.

  • There is no plugin for Vera that offers disaggregation.

  • Bidgely offers disaggregation, but doesn’t interface with Vera, and doesn’t work with Aeon Labs.

Is there any ray of sunshine that I’m missing here? I wasn’t expecting a miracle, but certainly some higher level of software for use with the Aeon Labs HEM & Vera. I guess I thought energy monitoring would be more central to the home automation experience at this point.[/quote]

I just built an infrared pulse meter to read my home energy meter and send the data to Vera (using Arduino and MySensors), and have been trying to find a way to get that information to a service that offers disaggregation.

I just discovered a service called PlotWatt this morning. As far as I can tell, it’s completely free for home use, and offers disaggregation as one of its main features. They have an API, and “PlotWatt API” is one of their energy meter types. In an email back to me, they stated “If you’re motivated, we’d love to have you share data using our api. FYI, our support for the residential hacker community is super limited.”

So, it sounds like we’re mostly on our own, but they encourage us to make use of their API. It’s well documented, from what I’ve seen. I have a decent bit of coding experience, but no Vera plugin experience. I’m going to start with guessed’s (that’s an awkward possessive…) SEG script, and try and take a look at the Ergy plugin, and see what I can do.

I recently put that SEG data-push logic directly inside my Spark. It meant that I could measure temperature, and put the device up in the attic (etc), and have it be autonomous (no Vera depends).

It’s feeding data to SEG directly, just as my RPi/Brultech and Vera units are all doing. It scales out better than going through a Hub, esp if you plan to add a lot of these.

Anyhow, you might consider having your Arduino device do that also, assuming it’s got direct Internet connectivity. I was going to do that for a water-meter reading device, when I have a few cycles.

[quote=“guessed, post:12, topic:181387”]I recently put that SEG data-push logic directly inside my Spark. It meant that I could measure temperature, and put the device up in the attic (etc), and have it be autonomous (no Vera depends).

It’s feeding data to SEG directly, just as my RPi/Brultech and Vera units are all doing. It scales out better than going through a Hub, esp if you plan to add a lot of these.

Anyhow, you might consider having your Arduino device do that also, assuming it’s got direct Internet connectivity. I was going to do that for a water-meter reading device, when I have a few cycles.[/quote]

I agree – on the Arduino itself probably would be a more ideal place for the uploading code. However, these are all non-ethernet Arduinos, radio-meshed and ultimately connected to the Vera via USB. I think the one upside to having the code on the Vera is that it abstracts the energy meter hardware layer; in Vera, all energy meters look pretty much the same, so hopefully if I get code working, other people can make use of it without needing an identical hardware setup.

Sure, just be aware that Vera (Vera 3) isn’t particularly stable once you have a reasonable update frequency on this data.

I was getting data pushed to me every 5 seconds from a 30+ channel Brultech GEM unit, and it would occasionally hit problems in Vera that would cause it to SEGV & restart… probably less likely to be hit if you’re low-volume / low-frequency but I definitely hit more Vera stability issues with the higher ingestion rates caused by having sensors.

Of course, YMMV. 8)

[quote=“guessed, post:14, topic:181387”]Sure, just be aware that Vera (Vera 3) isn’t particularly stable once you have a reasonable update frequency on this data.

I was getting data pushed to me every 5 seconds from a 30+ channel Brultech GEM unit, and it would occasionally hit problems in Vera that would cause it to SEGV & restart… probably less likely to be hit if you’re low-volume / low-frequency but I definitely hit more Vera stability issues with the higher ingestion rates caused by having sensors.

Of course, YMMV. 8)[/quote]

That’s good to know, thanks. Right now I’m pushing a single channel every 5 seconds, and I don’t plan on adding any more. I suspect I’d probably hit a limit sooner than you since I have the Vera Lite. If I do, I’ll dial back the update frequency.

I now have a functional script for uploading my power data to PlotWatt. I’ve been using it for less than a day, but it seems to be working well so far. Guessed, thanks for your SEG code, which gave me a great structure to modify for my script.

Here’s the thread.

Thanks guys! I’m following over at the other thread.