Luup Service IDs

[Buxton]

Hey AK,

I searched quite a bit for this topic with no real results. Here’s what I’m after.

I want to save the on/off state of my Zwave devices to a table, so that after a given event (that will change multiple device states), I can poll the table and restore the overall state of my zwave devices to what they were prior to the event. To do so, I’m thinking that I would iterate through the device table, and using the obtained device number, retrieve the status of the device using variable_get. However, programmatically getting the service IDs of devices is proving to be difficult. I imagine this is easy, but I’m not seeing it. Here is the code I have so far which gets me most of the way there:

[code]local idx = {}

for openLuupNo in pairs(luup.devices) do
if openLuupNo > 10000 then
idx[#idx+1] = openLuupNo
end
end
table.sort (idx)

for _, openLuupNo in ipairs(idx) do
local d = luup.devices[openLuupNo]
local devType = d.device_type
local devDescription = d.description
local devNodeID = d.id
local devString = (string.format(‘[%03d / %s] %s %s’, openLuupNo, devNodeID, devDescription, devType))

local devServiceID = "Some sort of transform of devType to devServiceID"
local w = luup.variable_get(devServiceID, "Status", openLuupNo)

if d.id > "1" then

– print (w)
print (devString)
end
end[/code]

As always, any help greatly appreciated.

[akbooer]

[ul][quote=“Buxton, post:1, topic:199487”]I want to save the on/off state of my Zwave devices to a table, so that after a given event (that will change multiple device states), I can poll the table and restore the overall state of my zwave devices to what they were prior to the event.[/quote]

I assume you have good reasons for this, but suggest that you are about to be skating on thin ice…

To do so, I'm thinking that I would iterate through the device table, and using the obtained device number, retrieve the status of the device using variable_get.

So far, so good.

However, programmatically getting the service IDs of devices is proving to be difficult. I imagine this is easy, but I'm not seeing it.

You are not missing anything, for there is nothing to see. It’s a sad fact that Vera/MiOS forgot to include a request which gives you what you want: “what serviceIds does this device support?” The nearest you can get using the luup.xxx() API is:
[list]
[li]luup.devices_by_service() - AFAIK this doesn’t work and has no useful documentation,[/li]
[li]luup.device_supports_service() - is actually the wrong way around. You need to test all devices and know what service you’re after.
[/li]
[/list]

The only other thing you can do on Vera, from a programmatic point of view would be to make an HTTP request to your own machine asking for the (huge) user_data file[/ul]

local _, user_data = luup.inet.wget "http://127.0.0.1:3480/data_request?id=user_data2"

…and then parse the user_data with a JSON decoder and search through all the device services.

Strangely, you CAN do what you want from a browser using the request:

http://127.0.0.1:3480/data_request?id=invoke

…but this returns HTML and requires you to follow device links to get at the data.

Here is the code I have so far which gets me most of the way there:

Two comments on this:

[ul][li]you shouldn’t really use the device boundary 10000 when searching for local devices, this is not a guaranteed number. The right way to do it would be to check that its parent (or higher ancestor) is not a VeraBridge.[/li]
[li]“Some sort of transform of devType to devServiceID” - this is where you fall through the ice, so to speak. There simply is no such unique mapping: serviceIds are totally separate from device types. However, if you do know your device types well, you know what services they generally provide. But code like this is likely to break.
[/li][/ul]

Even if you get to this point, how are you going to restore the old states? You can’t just change the variables in question, you have to go through a service action call, and you have to know what that is for each variable type that you are using.

HAVING SAID ALL THAT…

You are running on openLuup, and not Vera. It is, in fact, trivial to find the services and variables that belong to a device. This would, however, mean delving slightly below the thin veneer of inadequate luup.xxx() calls to the object-oriented layer which is the real heart of openLuup.

I will gladly tell you how to do that if you have an answer to the really difficult question of how you’re planning to restore the states.

[Buxton]

Vera/MiOS forgot

LOL! AK you are polite to a fault.

As for using a device boundary, I completely agree and it is not the way I’d like to go, but it seems to be the way most code handles obtaining ZWave only devices. Checking the parent works, (I believe the Info-Viewer plugin uses this method) but it seems overly complex for what I’m doing.

As for setting the device state, I intended to use this routine only for lights, so it would be relatively uncomplicated–using set_device. But I can imagine someone stumbling across this thread who might want to apply the same logic to other devices–say a group of thermostats in an event where there is a temporary break in the normal daily routine that requires a temperature adjustment, or even restoring values to non-zwave devices. In my case, a group of lights come on (depending on the time of day) triggered by a security event. When the event is cleared, the lights that came on (and were not on previously) are shut off from data supplied by a state table.

As you point out, both of these quandaries display weaknesses in the underlying vera luup engine. For example, one should simply be able to poll a device flag that would signify that it is indeed a zwave device. Likewise, one should also be able to easily determine the serviceids. So I’d like to propose that openLuup correct these deficiencies with a series of luup.xxx extensions.

When I was looking through the openLuup code in search of answers, I could make out the contours of a solution as the OOP model in Lua relies on tables, and accordingly, this type of data is stored in openLuup. But IMHO, making direct calls to the openLuup engine is also prone to failure as a change here or there could break the call. A luup extension , on the other hand, would offer relative simplicity to ones code, and safeguard, to a degree, against future disruption.

Anyway, my two cents for what it’s worth. If the above borders on heresy, I can certainly kluge a solution with some hard mapping. Let me know your thoughts and I’ll go forward accordingly.

[akbooer]

@Buxton

Thanks for your very thoughtful response. Much appreciated. Comments below…

@Buxton

As you point out, both of these quandaries display weaknesses in the underlying vera luup engine. For example, one should simply be able to poll a device flag that would signify that it is indeed a zwave device. Likewise, one should also be able to easily determine the serviceids. So I’d like to propose that openLuup correct these deficiencies with a series of luup.xxx extensions.

Well this, indeed, is my own quandary. I am very, very, loathe to make extensions to luup.xxx() calls. It’s only very recently that I’ve added a luup.openLuup flag to signify that you’re running on openLuup and not Vera. The whole point was that you shouldn’t be able to tell the difference. BUT… people always push things to the limit, and there are clearly some cases where you need to know this.

One option would simply to be to hang extensions off the, now extant, luup.openLuup branch. Thus you could have:

luup.openLuup.services (device_number)
luup.openLuup.actions (device_number)
luup.openLuup.variables (device_number)

Using luup.openLuup as a flag would still work if you test for non-nil, rather than true.

When I was looking through the openLuup code in search of answers, I could make out the contours of a solution as the OOP model in Lua relies on tables, and accordingly, this type of data is stored in openLuup. But IMHO, making direct calls to the openLuup engine is also prone to failure as a change here or there could break the call. A luup extension , on the other hand, would offer relative simplicity to ones code, and safeguard, to a degree, against future disruption.

As it is, the OO approach at the moment would be:

local device_number = 2
local d = luup.devices[device_number]
for serviceId, svc in pairs(d.services) do 
  print ("\n--- Actions for serviceId " .. serviceId)
  for action in pairs (svc.actions) do print (action) end
  print ("\n--- Variables for serviceId " .. serviceId)
  for variable, var in pairs (svc.variables) do print(variable, var.value) end
end
-- or your can just get to a list of variables by id directly from the device
print "\n--- Variables by id"
for id, var in ipairs(d.variables) do print(id, var.name, var.value) end

which, for me, gives:

--- Actions for serviceId openLuup
EmptyTrash
SetHouseMode
SendToTrash
Test
RunScene

--- Variables for serviceId openLuup
Memory_Mb 	8.6
HouseMode 	1
CpuLoad 	4.1
MemAvail_Mb 	816.5
MemTotal_Mb 	949.6
Version 	v18.7.15
Test 	123
MemFree_Mb 	655.9
StartTime 	2018-07-17T07:31:02
Uptime_Days 	0.15
Vnumber 	180715

--- Actions for serviceId urn:upnp-org:serviceId:altui1

--- Variables for serviceId urn:upnp-org:serviceId:altui1
DisplayLine1 	9 Mb, cpu 4.1%, 0.15 days
DisplayLine2 	[Home]

--- Variables by id
1 	HouseMode 	1
2 	DisplayLine2 	[Home]
3 	Version 	v18.7.15
4 	StartTime 	2018-07-17T07:31:02
5 	Memory_Mb 	8.6
6 	CpuLoad 	4.1
7 	Uptime_Days 	0.15
8 	DisplayLine1 	9 Mb, cpu 4.1%, 0.15 days
9 	MemFree_Mb 	655.9
10 	MemAvail_Mb 	816.5
11 	MemTotal_Mb 	949.6
12 	Vnumber 	180715

But you are absolutely right. This opens a whole new can of worms. The variable object is currently totally unprotected… don’t for one moment suspect that simply changing var.value to something else would correctly update the variable. The variable is the most complex structure in the system and has to handle callbacks, timestamps, user_data version numbers, data history, …

Anyway, my two cents for what it's worth. If the above borders on heresy, I can certainly kluge a solution with some hard mapping. Let me know your thoughts and I'll go forward accordingly.

Worth more than two cents, I would say. Perhaps I will upgrade you from Beta-tester to Alpha-tester?

[Buxton]

Yes, having luup extensions somewhat separated by the openLuup object, but nevertheless available, is a good way to maintain the integrity of the engine. It would also allow you to sandbox some of the internals where you see the necessity. So a good compromise, and documenting such additions would be more compact.

Alpha away, though like most, my attention span is limited by how chaotic my worklife is!

[Buxton]

One more similar aspect that came to mind (but I forgot about when writing the above) is obtaining scene data. This post touches on it: http://forum.micasaverde.com/index.php/topic,8537.msg54699.html#msg54699

Another way to accomplish saving a state table would entail using scenes as the foundation. Per what I wrote above, does openLuup have access to at least “native” AltUI scene data? I know you don’t bring across vera scene data during bridging, rather you create a reference to scenes with a simple http call to the remote vera.

For example, one would create a scene where specific devices are turned on or off during an event. During the event and prior to running the scene, save the current status of the devices that are encapsulated in the scene via a lua script. Then run the response to the event which includes firing the scene, and when the event clears, use the state table originally derived from the scene to reinstate the status of the various devices that may (or may not) have been effected. This allows a great deal of flexibility over what devices to use in an event response, and as well, the logic following an event’s aftermath.

The problem of course is that luup.xxx does not give you direct access to this type of data…

I promise this is my last post on the topic!

[akbooer]

@Buxton

does openLuup have access to at least “native” AltUI scene data?

In fact, scenes are stored and implemented natively in openLuup, not AltUI. The only feature of Vera scenes that I intentionally omitted was triggers, since AltUI does a much better job with device variable watches.

I know you don't bring across vera scene data during bridging, rather you create a reference to scenes with a simple http call to the remote vera.

That’s the default bridge behaviour, yes. But also, it’s possible to import Vera scenes and have them automatically converted to ‘native’ Lua scenes. VeraBridge’s GetVeraScenes action will do this for you.

For example, one would create a scene where specific devices are turned on or off during an event. During the event and prior to running the scene, save the current status of the devices that are encapsulated in the scene via a lua script. Then run the response to the event which includes firing the scene, and when the event clears, use the state table originally derived from the scene to reinstate the status of the various devices that may (or may not) have been effected. This allows a great deal of flexibility over what devices to use in an event response, and as well, the logic following an event's aftermath.

Interestingly, after some arm-twisting, forum member @DesT persuaded me to implement user-defined global callbacks before and after any scene is run. These are simply global functions defined in Lua Startup and activated by defining their names in system attributes openLuup.Scenes.Prolog and .Epilog.

The problem of course is that luup.xxx does not give you direct access to this type of data....

No, but, once again, at the openLuup OO layer, it’s all instantly available. By way of example, a method call on a scene object reveals for one of my scenes:

{
	 Timestamp = 1483028673,
	 favorite = false,
	 groups = {{
	 	 	 actions = {{
	 	 	 	 	 action = "SetTarget",
	 	 	 	 	 arguments = {{
	 	 	 	 	 	 	 name = "newTargetValue",
	 	 	 	 	 	 	 value = "1"
	 	 	 	 	 	 }},
	 	 	 	 	 device = "50",
	 	 	 	 	 service = "urn:upnp-org:serviceId:SwitchPower1"
	 	 	 	 }},
	 	 	 delay = 0
	 	 }},
	 id = 5,
	 last_run = 1532027621,
	 lua = "",
	 modeStatus = "0",
	 name = "Landing Table ON",
	 paused = 0,
	 room = 0,
	 timers = {{
	 	 	 days_of_week = "1,2,3,4,5,6,7",
	 	 	 enabled = 1,
	 	 	 id = 1,
	 	 	 last_run = 1532027621,
	 	 	 name = "new timer",
	 	 	 next_run = 1532113949.9436,
	 	 	 time = "-01:00:00T",
	 	 	 type = 2
	 	 }},
	 triggers = {},
	 triggers_operator = "OR"
}

You have everything: timers, delayed actions, etc. This is exactly the structure (although in Lua and not in JSON) defined here: http://wiki.micasaverde.com/index.php/Scene_Syntax

This could, of course, be wrapped as a function call in luup.openLuup.xxx

I promise this is my last post on the topic!

I do hope not. This turns out to be quite thought-provoking.