Thinking of building Record/Playback for Lighting (Vacation Mode Plugin)

Developer's Corner General Dev Discussions
1

After looking at the Energy Monitor codebase, I think there are enough “bits” to build a Vacation-Mode Plugin, something that records a few days worth of Lighting activity into a Calendar, and then can be set to “playback” these events.

I’ve done some experiments using [tt]curl[/tt] to PUT Calendar events into Google Calendar using CalDAV, and these are working fine.

Currently I’m experimenting using iCal entries like the following for the “Record” function:

[tt] BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//MiOS Vacation Plugin//CalDAV Client//EN
BEGIN:VEVENT
UID:20100712T182145Z-123401@example.com
DTSTAMP:20100613T202145Z
DTSTART:20100613T170005Z
DTEND:20100613T170100Z
SUMMARY:Set Kitchen Main Light to 50%
DESCRIPTION:A Test Calendar entry to be created by the Vera Vacation Plugin.\n\nBEGIN-MiOS\nluup.call_action(“urn:upnp-org:serviceId:Dimming1”, “SetLoadLevelTarget”, {newLoadlevelTarget=“30”}, 45)\n\END-MiOS
CLASS:PRIVATE
END:VEVENT
END:VCALENDAR
[/tt]

and I can’t make up my mind whether to use the “[tt]SUMMARY[/tt]” Field, with a human-readable command like:

[tt] Set Kitchen Main Lights to 50%[/tt] - For Dimmers
[tt] Set Living Picture Light to On[/tt] - For Switches/Dimmers
[tt] Set Master Bedroom Light to Off[/tt] - For Switches/Dimmers

which I’ll parse out and build the Vera command…

Or something like what I’ve done in the “[tt]DESCRIPTION[/tt]” Field, which contains a pure-lua snippet between [tt]BEGIN-MiOS/END-MiOS[/tt] Tagging:

[tt] A Test Calendar entry to be created by the Vera Vacation Plugin.\n\nBEGIN-MiOS\nluup.call_action(“urn:upnp-org:serviceId:Dimming1”, “SetLoadLevelTarget”, {newLoadlevelTarget=“30”}, 45)\n\END-MiOS[/tt]

The latter is much more flexible, since you could put just about any Lua/Luup code in there and have it execute (all Security issues aside :wink:

The former is much easier for folks to read, and potentially manually create, as long as the Device Descriptions are unique enough. If folks have non-unique Device Labels, then the syntax would need to be augmented with the DeviceID as in:

[tt] Set Kitchen Main Lights #54 to 50%[/tt] - For Dimmers

Since it’s going to chew up a bunch of time to write this, anyone have any opinions on which style they’d prefer to see?

Anyhow, my goal is to let Vera record, and then playback, events from a [remote] Calendar (Google, to start with, Yahoo next, and then any CalDAV server eventually). Letting it capture Lighting information (initially) so you can drive “away/vacation” schedules, and make your house look really lived in (since it will have followed real activity for a few days, recording all the Lighting “action” going on)

Anyone interested and/or have thoughts on it?

Once I get enough bits, I’ll request a code.mios.com space for it and my test files.

2

If anyone wants to do similar experiments, here’s the curl command I used to push the previous Calendar data to my GMail account (after enabling Calendaring for my GMail account):

curl --verbose --user "xxxxxxx@gmail.com:yyyyyy" --insecure --basic ^ --data-binary @test.ics --output test.txt --request PUT ^ --header "If-None-Match: *" ^ --header "Content-Type: text/calendar; charset="utf-8"; component=VEVENT" ^ --url https://www.google.com/calendar/dav/xxxxxxx@gmail.com/events/test1.ics

Where [tt]xxxxxxx[/tt] is your GMail userid, and [tt]yyyyyy[/tt] is your password for that account.

All tests were run under a Windows version of cURL, in a .bat file, but presumably the same stuff works on Vera also (since it uses cURL for a bunch of stuff)

3

Not sure about recording-for-playback, but playback from GCal would be extremely useful - assuming user will be able to type event details into GCal

Have you looked into RememberTheMilk service? IMHO it’s tailored for this, and they have rich API too.

4

Right, either way there’s a “record” half and a “playback” half of the code, with a shared repository in the middle (CalDAV Calendar).

Folks who don’t want to record, could manually type in the Calendar entries, in the right format, and just enable the Playback component. It’s likely that the Playback bit would be built first anyhow, but the experiments above were done to ensure that I could eventually “record”, since that will likely be simpler (and more realistic looking) than anything created by hand. If “manual” is going to be common, then the simple syntax below would be the better “first bet” for implementation.

The CalDAV choice was deliberate. It’s an open standard, which is run in common (free) hosted environments (eg, Google, Yahoo, etc) but can also be run on a locally-hosted server (various vendors) should people want tighter security (folks have already desire for better “off-net” options)

Additionally, it’s automatically exposed (editable) on an iPhone/iPad, and other phone brands, as well as via Thunderbird/Lightning, Sunbird, Web Browsers and even some plugins for Outlook.

This combo makes any “scheduling” activity very accessible to all the devices that anyone might consider running on.

5

I had thought about building this same functionality, but have not have the time to even design it out.

6

Some of the ideas I had include:

  1. Not limiting recording and playback to lighting, but support all actions on all devices. When in record mode the user would pick the devices and their actions that should be recorded.

  2. Having playback handled by a virtual device, so one could add multiple playback devices and playback multiple schedules. I was thinking that using script playback might be an easier way to schedule climate and light schedules than using scenes with timers. The would be particularly true if integrated with a CalDAV server, as it would be easier to see a full day/week/etc of scheduled actions.

7

The initial limitation will be lighting, but since I’ll check it into code.mios.com, folks will be able to extend it to cover other devices/device types.

My original prioritization was for things needed for “Vacation mode”, in this order:

1. Lighting (Dimmers and Regular Switches)
2. Scenes (for things like "All off")
3. Thermostats (since I just installed 2 :)

Folks should be able to add other types once the baseline functionality is in place.

Note, since there’s no UI to drive this within Vera, control will be extremely simple. It’ll be a Plugin, which can be instantiated as often as you like (Multiple Scheduler devices), each having the following characteristics:

An "On/Off" Switch to enable Record Mode
An "On/Off" Switch to enable Playback Mode
URL Parameter for CalDAV Account
Username/Password information for CalDAV Account

any filtering of Recorded data can be done manually by going into your favorite Calendar client and deleting stuff :slight_smile: You’ll also use your native calendar client to “move” any Recorded events to the right date range for Playback.

We also have to assume that both the Clock and TZ are setup correctly in Vera (similar to how the Weather Plugin assumes that Locale/Region is set correctly).

I was thinking that using script playback might be an easier way to schedule climate and light schedules than using scenes with timers. The would be particularly true if integrated with a CalDAV server, as it would be easier to see a full day/week/etc of scheduled actions.

Exactly, except that there’s no notion of Sunset/Sunrise in Calendar, so that Timer-type has no counterpart.

This will, however, start small being focussed mostly on Vacation-oriented stuff, but it should be possible to grow the codebase over time with more scheduling functionality.

I was thinking of having it “check in” with the Calendar about every 15-30 minutes in order to pickup the next 1 hr worth of “work”. This should cutdown on the overheads, but also means that “instant” scheduling wouldn’t work (this timing will be a parameter so people can tweak)

The next step is to work out how to get stuff “out” of Calendar, over CalDAV, including recurring events (etc).

Getting Calendar entries created was relatively easy… luckily the Mrs is away for a few weeks so I have time to research :wink:

JAVIER: Can you create me a code.mios.com space for a “Scheduler” Plugin? I can use this to at least upload the CalDAV test scripts prior to the Plugin’s creation.

8

sure, but ‘Scheduler’ is a little too generic. what about ‘CalDAV Scheduling’ or even ‘CalDAV storage’ ?

9

As a potential extension of the recording feature - human readable log of human-initiated events (lights on/off, thermostat manual changes, sensors) could be useful as part of “nanny surveillance system”. (Video is better, but only if one has a camera in every corner of the house, which is rarely the case.)

10

I would like to suggest decomposing your plugin into two components:

Component 1/Plugin 1 (only one instance allowed): Notification service
Clients (other plugins) subscribe to the notification service and get notified about events (light on/off, run scene, set dimmer, …).

Component 2/Plugin 2 (multiple instances allowed): CalDAV Scheduling
Uses Component 1/Plugin 1

11

Hey Javier,
Let’s shoot for:

 CalDAV Scheduler

BTW: Do you have a set of internal “rules” you go by for plugin naming? I’d like to avoid Apple-Store style rejections :wink:

12

@Ap15e,
No need to split it in two if all you want is notification of standard Device events within Vera. After looking at this code for a while:

http://code.mios.com/trac/mios_energry-reporting/browser/energy%20monitor%20plugin/L_EnergyMonitor_j.lua#L138

and then this method implementation:

http://code.mios.com/trac/mios_energry-reporting/browser/energy%20monitor%20plugin/L_EnergyMonitor_j.lua#L25

it finally dawned on me how [tt]luup.variable_watch[/tt] works. Basically it’s an event hook that lets you subscribe to changes in a named-variable, on a specified DeviceID (more or less).

Anyhow, this makes a much better example than the documentation.

For the CalDAV Scheduler, we’d simply subscribe to the ON/Off or Dimmer events on ALL lighting fixtures, similar to how the Energy Monitor works. Over time, we’d end up subscribing to various other Variables on other devices so they can be watched also (SetPoints and such for Thermostats etc)

So any Plugin that wants to see changes in standard Vera Devices can also just subscribe itself in a similar manner. No need to split this out, since it already exists are core Luup functionality.

Now if we ever wanted to make the CalDAV Scheduler do “stuff” on other devices, ones that dont already implement a standard service, all we’d need to do is implement a new Service for the Scheduler to look for using:

[tt] luup.device_supports_service[/tt]

and then it’ll call that. you could imagine an eventual syntax like:

[tt] Call SqueezeBox Living Room #58 with [/tt]

where this invokes some custom Service, defined by the CalDAV Scheduler, that passes an arbitrary string over a generic Service Action. The Plugins could do whatever they want with it, but it would be a step above calling a “Scene” since you’d be able to pass a Parameter.

But I’m getting ahead of myself here.

First I have to work out how to use CalDAV REPORT to form a query to extract the next hours worth of data :wink:

13

[quote=“guessed, post:11, topic:166042”]Let’s shoot for:

 CalDAV Scheduler[/quote]

done

not really. i have just been bitten too many times by overly broad names.

14

I’d like to create a plugin called “Stuff” (just kidding)

15

@guessed,

Thanks for pointing out the existence of luup.variable_watch!

With this function you are just a few lines of code away from a notification service:
Just iterate over the devices and subscribe to the relevant variables.

Just for reference: the formal specification is at:
http://wiki.micasaverde.com/index.php/Luup_Lua_extensions#function:_variable_watch

16

@Ap15e, you’re welcome, but it’s Javier’s code. I just hijacked it from code.mios.com, which is useful for code examples :slight_smile:

ok, so turns out we can Pull CalDAV data relatively simply using a “REPORT” request (HTTP “REPORT” method instead of PUT/POST etc) and a body containing a TZ selection, and a data-range block. There will be some “corner” cases to deal with, but overall not that big a deal.

For those interested in “experiments”, I checked in some of the samples I use to test (using a local copy of curl)

17

[quote=“Ap15e, post:15, topic:166042”]@guessed,

Thanks for pointing out the existence of luup.variable_watch![/quote]

I finally got around to playing with variable_watch… what a great tool. For once, I have written an all inclusive garage door opener plugin that doesn’t need scenes created to set status upon tripping/untripping of the sensor.

Just to confirm, I only need to invoke variable_watch in the plugin startup function, and it will run the called function every time the variable is changed, correct? It seems to work this way. Unlike call_timer, it doesn’t need to be called at the end of the called function, correct?

18

@woodsby, yes… that’s my understanding. You’re effectively registering a Callback handler, or observer, which need only be done once, at Startup.

I believe the Scene “Event” mechanism itself relies upon this same mechanism, and I believe you’ll also see [tt]LuaUPnP.log[/tt] entries where the “[tt]#hooks[/tt]” parameter is non-Zero, indicating a handler is hanging off it.

19

Thanks again… totally unrelated to the topic here, but what a great tool.

20

yeah, this one little item is both very handy, and very dangerous at the same time. It could do with some real doco, but you can basically work it out from the Energy Monitor Plugin.

All sorts of interesting stuff could be done with that hook, esp if you wanted to “mirror” bits of data in Vera with something else, in more of a “push” model (compared to the pull model of the URL-based stuff) … 8)