DataYours - monitoring energy usage

Plugins & Plugin Development DataYours
1

I’m investigating a ‘solution’ to the problem of monitoring energy usage with [tt]DataYours[/tt]. The basic requirements I am working towards are:

[ol][li]No need to individually select variables for power monitoring[/li]
[li]Usage aggregated into kWh per day / month / …[/li]
[li]Compatible with existing archives and plotting[/li]
[li]Possibility to use custom graphics in future[/li]
[li]Possibility to export to external services (like SEG, etc.)[/li][/ol]

As a start (1,2, and 3 above), I have modified [tt]DataWatcher[/tt] to use Vera’s [tt]live_energy_usage[/tt] reporting documented here: [url=http://wiki.mios.com/index.php/Luup_Requests#live_energy_usage]Luup Requests - MiOS. This report is polled every minute, the results converted to kWh per minute, and each value reported for each device with serviceId [tt]urn:micasaverde-com:serviceId:EnergyMetering1[/tt] and variable name [tt]EnergyUsage[/tt].

To accumulate the results correctly, data storage should be configured with an archive structure which starts with [tt]“1m:1d”[/tt] and an aggregation function of “sum”. A typical example would be [tt]“1m:1d, 1d:5y”[/tt]. This means that for the first day, the by-the-minute data is available, for any times greater than that (up to 5 years) the daily usage will be stored. If you wanted monthly reporting instead, use something like [tt]“1m:1d, 1d:30d, 30d:10y”[/tt].

In order to make this archive configuration automatic, a new rule should be added to each of the storage configuration files in the Whisper database directory:

storage-schemas.conf:

[EnergyUsage]
pattern = \.EnergyUsage
retentions = 1m:1d,1d:5y

storage-aggregation.conf:

[EnergyUsage]
pattern = \.EnergyUsage
xFilesFactor = 0
aggregationMethod = sum

Finally, to enable this feature, the new [tt]DataWatcher[/tt] configuration variable [tt]LIVE_ENERGY_USAGE[/tt] should be set to 1. The new Whisper database files should appear on the Whisper treemap, and can be plotted as usual. Any device which is reporting power usage will be flagged as being watched in the Devices display, although viewing the individual variables of any such device will not show that anything is being watched in the conventional sense (because it isn’t.)

The performance of this entirely relies on the accuracy of Vera’s live_energy_usage reporting, but I’ve been trying it out for a few days and it seems OK. A simple example of a daily and weekly report is attached below, along with the updated [tt]L_DataWatcher.lua[/tt] file.

I’d welcome any experiences of anyone trying this out. Thanks.

2

Hi akbooer,

how can I apply this power usage reporting to Datayours on Rpi ?

tnks

donato

3

Just update your [tt]L_DataWatcher7.lua[/tt] file on Vera and leave [tt]DataCache[/tt], [tt]DataGraph[/tt], and [tt]DataDash[/tt] running on the Rpi.

4

Just update your [tt]L_DataWatcher7.lua[/tt] file on Vera and leave [tt]DataCache[/tt], [tt]DataGraph[/tt], and [tt]DataDash[/tt] running on the Rpi.[/quote]

Don’t I have to change the storage schema and aggregation files ?

tnks donato

5

Oh yes, all those configuration instructions still apply, of course.

6

Oh yes, all those configuration instructions still apply, of course.[/quote]
Hi akbooer

excuse me for the stupid question : on the Rpi where can i find these files or where do i put them (in which folder)?

donato

7

Same as on a real Vera…

For convenience, I attach examples of these files (including the live energy usage modifications), but they may need to be changed to match your needs.

8

Hi akbooer,

You keep adding the nice stuff :-). Would this work for plug-ins that read energy meters or solar panel production as well, or only Z-Wave power meter devices? I just looked and one of my solar plugins show in the live_energy_usage request so I guess I could redesign the others to work as well. I was thinking of building this kind of daily/weekly/monthly/yearly view in those plug ins but if dataYours takes that problem away I would be happy to say that folks should install dataYours for nice reporting (with a Rpi to off load the reporting and UI).

Cheers Rene

9

Well, yes, I think it works for anything with a Watts variable, although I’m not quite sure.

I’ve realised that this code could, in fact, run anywhere. It doesn’t need to be on the same machine since it is accessed through HTTP. So I might refactor it some time and move it out of [tt]DataWatcher[/tt], perhaps a separate daemon, but the basic functionality will remain somewhere.

It turns out that a slight tweak to [tt]DataDash[/tt] is also required in some cases, because some of the usage might be from hidden devices (this is the case for Fibaro switches, for example) and, as it stands, such devices aren’t named in the GUI. It still works, but you just get device numbers, not names.

I can post an update is that’s needed by anyone to fix this.

10

akbooer

Thanks for all your hard work on this… Hope you can help me with an issue:

I am using a Aeon HEM and after I pair it I get 3 actual devices… so I am getting 3 graphs. How can I reduce that down to 1 graph with the total energy consumption for the entire house?

Thank you

11

This is not built-in functionality for DataYours. However, the Power Arithmetic plugin by @futzle (available from the MiOS App Store) is perhaps just the thing you need to sum the channels. The output total can then be logged by DataYours as a separate channel.

12

Thank you… i will look into that.

13

See also examples in this thread: http://forum.micasaverde.com/index.php/topic,15840.0.html

14

Hi Akboer,

I am using Datayours since february and it’s been working perfectly for the last couple of months. FYI, I am using a current cost pulse meter connected to Vera.

I run a csv report each week which gives me the total energy usage for that week in Kwh. I than manually compare the results by reading the display on my energy meter in the house. Usually the results match correctly between datayours and my energy meter. Sometimes there is a small difference however.

Here is my question: Is it possible to ‘synchronize’ the exact no. of Kwh on my energy meter with the results from datayours? This system is located in a remote location and hence the only way I can retrieve the meter results is by running a datayours query over the internet, (which works perfectly fine by the way).

Thanks a lot,

Jacques

15

That’s what we like to hear, thanks.

I am using a current cost pulse meter connected to Vera.

I run a csv report each week which gives me the total energy usage for that week in Kwh. I than manually compare the results by reading the display on my energy meter in the house. Usually the results match correctly between datayours and my energy meter. Sometimes there is a small difference however.

Here is my question: Is it possible to ‘synchronize’ the exact no. of Kwh on my energy meter with the results from datayours? This system is located in a remote location and hence the only way I can retrieve the meter results is by running a datayours query over the internet, (which works perfectly fine by the way).

This seems to me to be more of a question of the energy meter calibration itself, than a DataYours issue. All that DataYours does is report the values of changed variables, if those numbers are wrong, then too bad!

There is an important difference between pulse counters and Hall-effect probes. In the former, hardware (independent of Vera) does the integration, in the latter you have to do it in software. So it’s quite possible, if and when Vera goes down for a while and restarts, that there will be an increasing drift between the total energy accumulated and the actual value that a meter shows. However, this should not generally be an issue for a pulse counter, unless there is an issue with the initial setup and calibration.

16

In the csv reports I have indeed noticed that about 20 days over a 6 months period had ‘nil’ values. Could this be because the vera was down at these days? Again this is a holiday home so I am not always there.

I will also try to re-calibrate my current cost device to see if that helps.

Thanks again.

Jacques

17

Need more data. It depends on the archive structure you have set, but it would be surprising if a whole day went by without recording anything. What is the refresh rate from the meter supposed to be? It should be able to restart and all that happens is you lose the update for that particular time period, not the whole day.

18

This is great, and looking to use instead of SEG now they are charging.
Can I set this up so I can see more granular, maybe 30 min for every day for a month, then daily for 5 years for example? Would better help show behaviour and useage through the day as to what is causing the power usage.

19

Sorry to be slow in replying… I missed this post entirely.

Yes, you can set up an archive for exactly that “1m:1d, 30m:30d, 1d:5y”

20

Thanks. But that involves deleting the current archive and starting again. Correct?