Finding out which devices and scenes affect one another

To help analyse scene/device interaction, I use this piece of code which scans all scenes for connected devices: both devices which trigger the scene, and devices which are actioned by the scene. It writes a file /www/scene_devices.txt which can be examined from a web browser using http://your-vera-ip-address/scene_devices.txt and gives an output like:

SCENES: 

 [2] "Barn All Off" 
  actions = {[151] Far Beam, [172] Near Beam, [173] Chandelier, [178] Standard Lamp, [185] Kitchen, [190] Dining Table}

 [3] "Good Night" 
  actions = {[146] Bedside 1, [147] Bedside 2, [187] Bedroom Ceiling, [199] Table Lamp}

 [5] "Apple TV Off" 
  actions = {[225] Apple-TV-Barn}

 [89] "TEST" 
  + Lua code 
  triggers = {[181] Study Ceiling}
  actions  = {[268] Smtp Notification}

Each scene has a scene number/name, as does each device. I’ve found this helpful in tracking down scene-related errors. What it can’t do is detect any interactions between devices and Lua code (in scenes, triggers, or other devices like PLEG), although it does flag the presence of Lua code attached to scenes.

Code appended below. Simply cut and paste into “Test Luup code” and press go, then head to another browser window to inspect the result. You could also put it into a scene’s Luup code and run it on demand. Note that there is one dependency: it requires a json package - this may involve a change to the first line json = require “json”. For testing I use the pure-Lua package dkjson from [url=http://dkolf.de/src/dkjson-lua.fsl/home]dkjson - dkjson, but if you have an app which already loads its own json package you may be able to use that (for example, dataMine has one, so writing json = require “json-dm” works just fine.)

Hope this helps.

json = require "json"

-- Build list of scenes with trigger devices and actioned devices
--
-- version = "2013.06.20 @akbooer"
--

-- SET and LIST utility functions: map, flatten, and set

-- classic map utility - cf. pairs
local function map (Xs , fct)   -- map function to each item in table, returns {} if none
	local table = {}
	for i,x in pairs (Xs or {}) do table[i] = fct(x) end
	return table 
end

-- classic list flatten
local function flatten (array)
	local l = {}
	local function add_item (x) l[#l+1] = x; end
	for _,x in ipairs(array) do
		if type(x) == "table" then
			map(flatten(x), add_item )
		else
			add_item (x)
		end
	end
	return l
end

-- set operations, only what we need here: add, list 
local function set()								-- create new empty set
	local s = {}		-- holder for set
	return {
		add  = function (x) s[x] = x; end, 			-- add element to the set
		list = function ( ) 						-- return sorted list of set elements
			local l = {}; 
			for i in pairs(s) do l[#l+1] = i; end; 
			table.sort (l);
			return l; end
		}
end

-- formatting functions for printing data structure
local function format_devices(d)
	return map (d, function (x) return string.format ('[%d] %s', x, (luup.devices[tonumber(x)] or {description = ''}).description); end)
end

local function format_triggers (t)
	if #t == 0 then return '' end
	return table.concat {'  triggers = {', table.concat(format_devices(t), ', '), '}\n' }
end

local function format_actions (t)
	if #t == 0 then return '' end
	return table.concat {'  actions  = {', table.concat(format_devices(t), ', '), '}\n' }
end

local function format_lua (lua)
	if not lua then return '' end
	return '  + Lua code \n'
end

local function scene_tostring (s)
	return  string.format('[%d] "%s" \n%s%s%s ', s.id, s.name, format_lua(s.lua), format_triggers(s.triggers), format_actions(s.actions) )
end

-- functions to build data structure of devices triggering scenes, and actioned by scenes
local function get_device (x) return x.device end

local function trigger_devices (t)
	local trigger_set = set()  					-- create a new empty set (of device numbers)
	local device_list = map (t, get_device)		-- create list of device numbers
	map (device_list, trigger_set.add)			-- add elements to set
	return trigger_set.list ()					-- return sorted list of set members
end

local function action_devices (groups)
	local action_set = set() 					-- create a new empty set (of device numbers)
	local function actions (g) return map (g.actions, get_device) end
	local action_list = map (groups, actions)	-- create list of actions
	map (flatten(action_list), action_set.add)	-- mash together all the action device lists
	return action_set.list ()					-- return sorted list of set members
end

local function attached_devices (sceneNo) 
	local _, s = luup.inet.wget("http://127.0.0.1:3480/data_request?id=scene&action=list&scene=" .. sceneNo)
	if s == "ERROR" then return end
	
	s = json.decode(s)

	return setmetatable ({
		id   = s.id,
		name = s.name,
		lua  = not not s.lua,			-- double negative ensures boolean return, not Lua string itself! 
		triggers = trigger_devices (s.triggers),
		actions  = action_devices  (s.groups) ,
		},
		{ __tostring = scene_tostring} )
end

-- main: build and write the database

local filename = "/www/scene_devices.txt"
local file = io.open(filename, "w")

if file then
	luup.log ("Opening: " .. filename)
	file:write '\nSCENES: \n\n '

	for i in pairs (luup.scenes) do
		scene_devices = attached_devices (i)
		file:write( tostring(scene_devices) )
	end
	
	file:close()
	luup.log ("Closing: " .. filename)	
else
	luup.log ("Failed to open: " .. filename)
end

Hey akbooer. This sounds very interesting, but I pasted it into my test luup code and run it and I got an empty scene_devices.txt. It creates the file, but empty. Any clue?

Nice … this should also be a standard Vera Feature on the Automation tab!

Sorry you’re having a problem with this.
My guess would be that the JSON module is not being correctly accessed.

[ul][li]Do you have dataMine installed? [/li]
[li]Did you follow the instructions on configuring the JSON package in the original post?[/li]
[li]What does the log file say when you run the code?[/li][/ul]

I followed this steps.

[ol][li]I downloaded dkjson.lua from dkjson - dkjson
[li]I uploaded that file to Vera via Develop Apps, Luup Files and restarted the engine[/li]
[li]I changed from your code json = require “json” to json = require “dkjson”[/li]
[li]Pasted the new code to the Test Luup Code (Lua)[/li]
[li]Went to http://my.veras.ip/scene_devices.txt and the file exists, but it is empty[/li][/ol]

Regards!

I think it’s not quite that simple to port modules to Vera since MCV has changed the ‘require’ code to cope with encoded files.

You easiest bet is certainly to install dataMine (you know you want to) and try again, having restored the code to reference json-dm.

Alternatively, I can post a far faster JSON module which will work too.

Finally I did it installing dataMine, but was not my first idea, if you have the JSON module, I will appreciate. Nice code! Thanx, congrats.

You can install DataMine … than delete the plugin … the files will still be available.
They do eat up some disk space.

Thanks, I am trying datamine and it may be a keeper jeje.

@akbooer, is it too difficult for your code to also include the lua code inserted on each scene?

Regards.

No, not at all. This is actually a slimmed-down version of a much more comprehensive scene formatter which I use and which decodes everything. I think for the simpler version, I’ll add the names of schedules and the Lua code and then post it. I was just trying to keep it concise and, in the first instance, the connected devices were the points of interest.

Here’s the modified code to list schedule names and Lua code as well:

-- Build list of scenes with trigger devices and actioned devices
--
-- version = "2013.11.14 @akbooer"
--

json = require "json" 

-- sceneBlog (), build list of scenes with trigger devices and actioned devices
-- as posted to http://forum.micasaverde.com/index.php/topic,15360.msg116759.html#msg116759
local function sceneBlog (filename)
	
	-- classic map utility - cf. pairs
	local function map (Xs , fct)   -- map function to each item in table, returns {} if none
		local table = {}
		for i,x in pairs (Xs or {}) do table[i] = fct(x) end
		return table 
	end
	
	-- classic list flatten
	local function flatten (array)
		local l = {}
		local function add_item (x) l[#l+1] = x; end
		for _,x in ipairs(array) do
			if type(x) == "table" then
				map(flatten(x), add_item )
			else
				add_item (x)
			end
		end
		return l
	end
	
	-- set operations, only what we need here: add, list 
	local function set()								-- create new empty set
		local s = {}		-- holder for set
		return {
			add  = function (x) s[x] = x; end, 			-- add element to the set
			list = function ( ) 						-- return sorted list of set elements
				local l = {}; 
				for i in pairs(s) do l[#l+1] = i; end; 
				table.sort (l);
				return l; end
			}
	end
	
	-- formatting functions for printing data structure
	local function format_devices(d)
		return map (d, function (x) return string.format ('[%d] %s', x, (luup.devices[tonumber(x)] or {description = ''}).description); end)
	end
	
	local function format_names(d)
		return map (d, function (x) return x.name or ''; end)
	end
	
	local function format_timers (t)
		t = t or {}
		if #t == 0 then return '' end
		return table.concat {'   schedules = {', table.concat(format_names(t), ', '), '}\n' }
	end
	
	local function format_triggers (t)
		if #t == 0 then return '' end
		return table.concat {'   triggers = {', table.concat(format_devices(t), ', '), '}\n' }
	end
	
	local function format_actions (t)
		if #t == 0 then return '' end
		return table.concat {'   actions = {', table.concat(format_devices(t), ', '), '}\n' }
	end
	
	local function format_lua (lua)
		if not lua then return '' end
		return '   Lua code: \n' .. lua 
	end
	
	local function scene_tostring (s)
		return  ("\n[%d] '%s' \n%s%s%s%s "): format (s.id, s.name, 
					format_timers(s.timers), format_triggers(s.triggers), format_actions(s.actions), format_lua(s.lua) )
	end
	
	-- functions to build data structure of devices triggering scenes, and actioned by scenes
	local function get_device (x) return x.device end
	
	local function trigger_devices (t)
		local trigger_set = set()  					-- create a new empty set (of device numbers)
		local device_list = map (t, get_device)		-- create list of device numbers
		map (device_list, trigger_set.add)			-- add elements to set
		return trigger_set.list ()					-- return sorted list of set members
	end
	
	local function action_devices (groups)
		local action_set = set() 					-- create a new empty set (of device numbers)
		local function actions (g) return map (g.actions, get_device) end
		local action_list = map (groups, actions)	-- create list of actions
		map (flatten(action_list), action_set.add)	-- mash together all the action device lists
		return action_set.list ()					-- return sorted list of set members
	end
	
	local function attached_devices (sceneNo) 
		local _, s = luup.inet.wget("http://127.0.0.1:3480/data_request?id=scene&action=list&scene=" .. sceneNo)
		if s == "ERROR" then return end
		
		s = json.decode(s)
		s.triggers = trigger_devices (s.triggers)		-- restructure slightly
		s.actions  = action_devices  (s.groups) 
	
		return setmetatable (s, { __tostring = scene_tostring} )
	end
	
	-- sceneBlog ()

	local file = io.open(filename, "w")
	
	if file then
		luup.log ("Opening: " .. filename)
		file:write '\nSCENES: \n\n '
	
		for i in pairs (luup.scenes) do
			local scene_devices = attached_devices (i)
			file:write( tostring(scene_devices) )
		end
		
		file:close()
		luup.log ("Closing: " .. filename)	
	else
		luup.log ("Failed to open: " .. filename)
	end	
end

-- main: build and write the database

sceneBlog "/www/scene_devices.txt"

…don’t forget that the ‘require’ near the start may need to be changed to your favourite JSON module.

Works beautifully! Thanx.

Maybe a final request would be one script that gives all the mapping of the scene, not just the devices involved but the actions and delays, like a report to have a backup of the whole scenes.

OK, here it is. Same caveat about the JSON ‘require’ statement as previously. The code just does one scene at a time, but you can call it as many times as you like…

-- sceneDetail(n)
--
-- write to a file the detail of scene N, in plain English

local json = require "akb-json"

local function sceneDetail (sceneName, filename)

	
	-- utility functions
	
	-- count members in table: works like #, but for non sequential elements
	local function count (table)
		local n = 0
		for _,_ in pairs (table or {}) do n = n + 1 end
		return n
	end;
	
	-- classic map utility - cf. pairs
	local function map (Xs , fct)   -- map function to each item in table, returns {} if none
		local table = {}
		for i,x in pairs (Xs or {}) do table[i] = fct(x) end
		return table 
	end;
	
	-- classic map utility - cf. ipairs
	local function imap (Xs , fct)   -- map function to each item in table, returns {} if none
		local table = {}
		for i,x in ipairs (Xs or {}) do table[i] = fct(x) end
		return table 
	end;
	
	
	-- useful resources elsewhere:
	--	luup.scenes: { {room_num = N, description = "", hidden = boolean}, ... } -- also OnDashboard and Timestamp
	--		documentation at: http://wiki.micasaverde.com/index.php/Scene_Syntax
	--		{id = N, name = "", room = N, groups = {...}, timers = {...}, triggers = {...} }
	--			group =  {delay = N, actions = {...} }
	--				action = {device = N, service = "", action = "" arguments = {...} } 
	--					argument = {name = "", value = "" }
	--			trigger = { name = "", enabled = N, template = N, device = N, arguments = {...} } -- cf. device json eventList
	--				argument = {id = N, value = "" }
	--			timer = {id = N, name = "", type = "", enabled = N, days_of_week = "", time = "", interval = "" }
	--
	
	-- formatting routines for various parameters
	
	local format = string.format
	
	local function format_devname (d) return (luup.devices[tonumber(d)] or {description = "no device name"}).description; end
	local function format_device  (d) return format ('[%s] %s', d, format_devname (d) ) end
	local function format_enabled (e) return ({[false] = "[OFF]", [true] = "[ON] "})[e.enabled] end
	local function format_room    (r) return format ('[%d] %s', r, (luup.rooms[r] or 'no room') ) end	
	local function format_date    (t) if t and t ~= "" then return os.date("%d-%b-%Y %X", t) end return '?' end	
	local function format_time    (t) return format ('"%s"', (t.time or '') ) end
	local function format_lua     (l) if l then return format('\n\nLua scene code: \n{\n%s}', l) else return '' end; end
	
	-- __tostring functions, mapped to various scene components
	
	local function trgarg_tostring   (x) return format ('%s', x.value or '') end
	local function actarg_tostring   (x) return format ('%s = %s', x.name, x.value) end
	local function plist_tostring	 (a) return format ('(%s)', table.concat (map(a, tostring) , ', ') )  end   -- list with ()
	local function blist_tostring	 (a) return format ('{%s}' ,table.concat (map(a, tostring) , ', ') )  end   -- list with {}
	
	local function interval_tostring (i) return format ('interval = "%s"', i.interval) end
	local function week_tostring     (w) return format ('days_of_week = {%s}, time = %s',  (w.days_of_week  or ''), format_time(w) ) end	
	local function month_tostring    (m) return format ('days_of_month = {%s}, time = %s', (m.days_of_month or ''), format_time(m) ) end
	local function absolute_tostring (a) return format ('absolute = %s', format_time(a) ) end
	
	local function timer_tostring (x)
		return format ('   %s "%s" (%s), Last run = %s, Next run = %s', 
			format_enabled(x), x.name, tostring(x.timer), format_date(x.last_run), format_date(x.next_run) )
	end
	
	local function trigger_tostring(x)
		return format ('   %s "%s" %s, {template = %s} %s %s', 
			format_enabled(x), x.name, format_device(x.device), x.template, tostring(x.arguments or ''), format_lua(x.lua) )
	end 
	
	local function action_tostring(x)
		return format('   [d=%d] %s.%s %s %s', x.delay, x.service:match "%w+$",   -- strip off uninteresting prefix 
			x.action, tostring(x.arguments or ''), format_device(x.device) )
	end	
	
	local function map_tostring      (x, txt) return format ('\n\n%s: (%d) \n%s \n', txt, #x, table.concat(map(x, tostring), '\n' ) ) end
	local function timers_tostring   (x) return map_tostring(x, 'Schedules') end
	local function triggers_tostring (x) return map_tostring(x, 'Triggers' ) end
	local function actions_tostring  (x) return map_tostring(x, 'Actions'  ) end    
	
	local function scene_tostring(x)
		return format ('Scene #%s: "%s", room: %s [%s] %s %s %s %s \n', 
			x.id, x.name, format_room(x.room), format_date(x.timestamp), format_lua(x.lua),
				tostring(x.timers), tostring(x.triggers), tostring(x.actions) )
	end 	
	
	
	-- OBJECTS for scene components	
	
	local function with_metamethod (table, fct)   -- returns table with "__tostring" metamethod attached, returns nil if table nil
		if table then
			return setmetatable (table ,{__tostring = fct}) 
		end
	end
	
	local function trgarg (arg)    -- trigger argument
		return with_metamethod ( 
			{
				id = arg.id, 
				value = arg.value
			}, 
		trgarg_tostring )
	end
	
	local function actarg (arg)   -- action argument
		return with_metamethod ( 
			{
				name = arg.name,
				value = arg.value
			}, 
		actarg_tostring )
	end
	
	-- timer subtypes
	local function interval (i) 
		return with_metamethod (
			{
				interval = i.interval
			}, 
		interval_tostring )	
	end
		
	local function weekly(w) 
		return with_metamethod (
			{
				days_of_week = w.days_of_week, 
				time = w.time
			}, 
		week_tostring )
	end
	
	local function monthly (m) 
		return with_metamethod (
			{
				days_of_month = m.days_of_week, 	-- note change of variable name to indicate month days, not weekdays
				time = m.time
			}, 
		month_tostring ) 
	end
	
	local function absolute (a)
		return with_metamethod (
			{
				time = a.time
			}, 
		absolute_tostring )
	end
	
	local function timer (t)
		return with_metamethod ( 
			{
				name = t.name,
				enabled = tonumber(t.enabled) == 1,   -- make boolean, rather than 0/1
				last_run = t.last_run,
				next_run = t.next_run,
				timer = ( {interval, weekly, monthly, absolute} ) [tonumber(t.type)] (t)   -- timer structure depends on type variable
			}, 
		timer_tostring )
	end
	
	local function trigger (t)
		return with_metamethod (
			{
				name = t.name,
				enabled = t.enabled == 1,   -- make boolean, rather than 0/1
				lua = t.lua,
				device = t.device,
				template = t.template,
				arguments = with_metamethod (imap (t.arguments, trgarg), plist_tostring)   -- nb. orderered list, hence imap
			}, 
		trigger_tostring )
	end
	
	local function action(a)
		return with_metamethod (
			{
				device  = a.device,
				service = a.service,
				delay = 0,				-- default value, updated later
				action =  a.action,
				arguments = with_metamethod (imap (a.arguments, actarg), blist_tostring)  -- nb. orderered list, hence imap
			},
		action_tostring )
	end
	
	-- flatten the JSON 'groups/actions' structure into actions with individual delays
	local function map_groups (groups)
		local delay   			-- delay of current group
		local list = {}   		-- accumulator for actions
	
		local function action_with_delay(a) a = action(a); a.delay = delay; list[#list + 1] = a; end	-- save delay as side effect
		local function group(g) delay = g.delay; map (g.actions, action_with_delay) ; end				-- store delay as side-effect
		map (groups, group)  
		
		return list  -- without metamethod, because timers and triggers also return from map without one
	end
	
	local function scene_data (sceneNo)   
		local _, s = luup.inet.wget("http://127.0.0.1:3480/data_request?id=scene&action=list&scene=" .. sceneNo)
		if s ~= "ERROR" then
			return json.decode(s)
		end
	end
	
	
	-- generic reverse lookup table constructor for element table.field
	local function lookupTable (table, field)
		field = field or "description"		-- default field name
		local lookup = {}  
		for i,d in pairs (table) do  		-- create the lookup table
			lookup [d[field] ] = i
		end
		return lookup
	end
	
	local sceneLookupTable				-- populated on the first call to sceneNo
	
	-- sceneNo (sceneName) returns matching scene number
	local function sceneNo (name)
	    sceneLookupTable = sceneLookupTable or lookupTable (luup.scenes) 
		return sceneLookupTable[name]
	end
	
	
	-- get(sceneNo) contructs an object representation of sceneNo 
	-- with lists of timers, triggers, actions, and lua objects
	-- and other information current at the time the object was created
	-- adds metables with __tostring function to enable text representation
	-- refactors the JSON 'groups/actions' structure into actions with individual delays
	local function getScene (sceneNo)	
		local x = scene_data (sceneNo)   
		if not x then return end
		return with_metamethod (
			{
				id   = x.id,
				name = x.name, 
				room = x.room,
				lua  = x.lua,
				timestamp = x.Timestamp,
				
				timers    = with_metamethod ( map (x.timers,   timer),   timers_tostring   ),
				triggers  = with_metamethod ( map (x.triggers, trigger), triggers_tostring ),
				actions   = with_metamethod ( map_groups (x.groups),	 actions_tostring  ),	
			},
		scene_tostring )
	end 
	
	-- sceneDetail ()

	local file = io.open(filename, "w")
	
	if file then
		local N = sceneNo (sceneName)
		luup.log ("Opening: " .. filename)
		file:write '\nScene Details: \n\n '	
		file:write( tostring(getScene (N)) )		
		file:close()
		luup.log ("Closing: " .. filename)	
	else
		luup.log ("Failed to open: " .. filename)
	end	
end

-- main: write the info for requested scene

sceneDetail ("Weekly kWh", "/www/scene_details1.txt")
sceneDetail ("Denon On", "/www/scene_details2.txt")

The last two lines call the function [tt]sceneDetail()[/tt] twice, by way of example, for two different scenes in my system. The first parameter is the name of the scene, the second is the name of the file to write.

The two file outputs are:

Scene Details: 

 Scene #94: "Weekly kWh", room: [0] no room [29-Jul-2013 15:22:24] 

Lua scene code: 
{
local kWh = AKB.NorthQ.get "Power Meter"
AKB.email.send ('Weekly kWh = ' .. kWh)

} 

Schedules: (1) 
   [ON]  "Weekly meter" (days_of_week = {6}, time = "9:0:0"), Last run = 09-Nov-2013 09:00:00, Next run = 16-Nov-2013 09:00:00 
 

Triggers: (0) 
 
 

Actions: (0) 
 

and:

Scene Details: 

 Scene #108: "Denon On", room: [13] A/V Equipment [03-Nov-2013 17:34:23]  

Schedules: (0) 
 
 

Triggers: (1) 
   [ON]  "Denon On" [234] Denon AVR-X2000, {template = 1} (1)  
 

Actions: (1) 
   [d=0] VSwitch1.SetTarget {newTargetValue = 1} [270] Denon Mimic 
 

Even this is not really a complete description - in particular the device triggers refer to service templates in the triggering device and are therefore a bit cryptic. It also does it best with the four different possible schedule types. If you need a complete description, then I’m afraid you are down to the raw JSON code from Luup request.

Hope this works for you OK.