GENA (General Event Notification Architecture)

Developer's Corner General Dev Discussions
#1

Does MiOS support [tt]GENA[/tt] (as required by the ‘UPnP™ Device Architecture 1.1’, section 4)?

#2

Some additional information:

Testbed:
SBS 7.6.x, Squeezeplay Beta, Whitebear Media Server (WMS) 2.3.0.2626 (port 31416) - running on 192.168.x.y
VeraLite 1.5.346 - running on 192.168.a.b

Setting up a GENA callback handler:

luup.register_handler( "GENA_callback", "GENA_callback" )
 
function GENA_callback( lul_request, lul_parameters, lul_outputformat )

 luup.log( 'GENA#1: ' .. tostring( lul_request ) )

 for k,v in pairs( lul_parameters )
  do
   luup.log( 'GENA#2: ' .. tostring( k ) .. '##' .. tostring( v ) )
  end

 luup.log( 'GENA#3: ' .. tostring( lul_outputformat ) )

 return 'OK'

end

Testing the GENA callback handler:

http://192.168.a.b:3480/data_request?id=lr_GENA_callback&Para1=Test1

Requesting a subscription to [tt]AVTransport[/tt] events:

telnet 192.168.x.y 31416

SUBSCRIBE /534d4257-3030-3064-3563-316566336635/AVTransport/eventing HTTP/1.1
TIMEOUT: Second-30
HOST: 192.168.x.y:31416
CALLBACK: <http://192.168.a.b:3480/data_request?id=lr_GENA_callback>
NT: upnp:event

GENA response from WMS:

HTTP/1.1 200 OK
Connection: close
Content-Length: 0
Date: Sun, 08 Apr 2012 16:24:40 GMT
SID: uuid:8d46dcc2-a2e5-4318-b101-2e6f178d1e46
Timeout: Second-30
Server: Whitebear/2.3 UPnP/1.0 Delphi-UPnP-Components/1.7

Problem:
WMS calls the callback URL, but [tt]lighttpd[/tt] doesn’t support the [tt]NOTIFY[/tt] method.

[code]**** 12:56:42.781 GENA Notification v2.3.0.2626 ****
HTTP/1.1 400 Bad Request
Content-Type: text/plain
Content-Length: 54
Connection: close

Error 400: Bad Request
Can not parse request: [NOTIFY][/code]

#3

Updating this public thread with more details on the issue so that the thread can be used as a reference in a Bug report against MCV.

Further findings…
In Lighttpd, it’s because of this:
http://redmine.lighttpd.net/projects/lighttpd/repository/entry/trunk/src/connections.c#L210

They don’t have cases for [tt]NOTIFY[/tt], [tt]MKCALENDAR[/tt], etc. If you’re not one of the core set, then you get rejected.

But it’s not just [tt]lighttpd[/tt] that has the issue. Once you get past [tt]lighttpd[/tt], you’d hit the Vera C Process ([tt]LuaUPnP[/tt]). This is what’s actually listening “behind” everything for the internal calls, with [tt]lighttpd[/tt] proxying calls through to it.

In this case, [tt]LuaUPnP[/tt] is linking in code from mongoose. Mongoose is the one that’s giving out the error in the case cited. You can tell by the error message it emits for the HTTP-400.

You can see the problem in this older rev of the file (which looks like what MCV is using)
http://mongoose.googlecode.com/svn/tags/2.0/mongoose.c

and the newer version:
mongoose/mongoose.c at master · cesanta/mongoose · GitHub

At this time, the only work-around to listen for standard UPnP NOTIFY Messages would be to write a free-standing Lua process to do the bridging work.

Having this functionality would allow AV Gear (and others) to real-time notify Vera of changes. Things like Channel changes, volume changes (etc) without need for heavyweight polling. In turn these Event could be used as Scene triggers for things like Lighting, or other general presence-based scenes.

#4

Bug 2529 filed to track.

#5

+1.

Guessed/Ap15e/MCV/McFlorin - Please accept this particular post as my support (as a user of and investor in the Vera/MIOS product) to prioritise the system’s ability to support GENA.

#6

[quote=“parkerc, post:5, topic:171160”]+1.

Guessed/Ap15e/MCV/McFlorin - Please accept this particular post as my support (as a user of and investor in the Vera/MIOS product) to prioritise the system’s ability to support GENA.[/quote]

DITTO

#7

[quote=“parkerc, post:5, topic:171160”]+1.

Guessed/Ap15e/MCV/McFlorin - Please accept this particular post as my support (as a user of and investor in the Vera/MIOS product) to prioritise the system’s ability to support GENA.[/quote]

Same as above!

#8

[quote=“parkerc, post:5, topic:171160”]+1.

Guessed/Ap15e/MCV/McFlorin - Please accept this particular post as my support (as a user of and investor in the Vera/MIOS product) to prioritise the system’s ability to support GENA.[/quote]

+1

#9

That’s exactly what I’ve done for the Belkin WeMo plugin. Take a look in the upnp-event-proxy.zip file. It should be generic enough for any plugin to use. There are comments at the start of the file, and of course there is working client code in the WeMo plugin itself.

I’m hoping to collaborate with other developers who need this functionality so that we can make a de facto standard UPnP event listener for all Vera plugins.

#10

Is it something we could use for the sonos plugin ?
Can you explain in few words the principles ?
How do we register to event notification ?

#11

You bet. The media controller plugins were front-and-centre in my mind when I created the proxy.

Can you explain in few words the principles ? How do we register to event notification ?

The proxy process starts at Vera startup (using code I haven’t written yet). It listens to TCP connections on port 2529. There is a RESTful HTTP interface which is used by both the client (your plugin) and the device (the UPnP device that the plugin controls). The usual sequence is:

[ol][li]The plugin does an HTTP GET to the proxy at localhost:2529 to check that the proxy is running.[/li]
[li]The plugin tells the device with a UPnP SUBSCRIBE message that it wants to be notified of events, with a callback URL of Vera’s port 2529. The device responds to the plugin with a subscription ID (SID)[/li]
[li]The plugin does an HTTP PUT to the proxy, telling the proxy the SID, and which Vera device+serviceId+action to call when it gets an event notification.[/li]
[li]Time passes…[/li]
[li]The device decides to notify subscribers of an event, and one of the subscribers is the proxy on port 2529, so the proxy gets a UPnP NOTIFY event.[/li]
[li]The proxy parses the NOTIFY message and pulls out the SID, the variable and the value.[/li]
[li]The proxy tells the Vera plugin of the event using HTTP lu_action to port 3480, using the action registered in step 3.[/li]
[li]The plugin runs the nominated action.[/li][/ol]

Later on, the plugin will need to renew the subscription by repeating the steps, since UPnP subscriptions have built-in obsolescence. The proxy doesn’t distinguish initial subscriptions from renewals, it just passes them on to the plugin the same.

I’ve commented the code liberally, so do look there too.

#12

I have no idea how to manually start a lua program (upnp-event-proxy.lua). Can you help ?

The usual sequence is:

[ol][li]The plugin does an HTTP GET to the proxy at localhost:2529 to check that the proxy is running.[/li]
[li]The plugin tells the device with a UPnP SUBSCRIBE message that it wants to be notified of events, with a callback URL of Vera’s port 2529. The device responds to the plugin with a subscription ID (SID)[/li]
[li]The plugin does an HTTP PUT to the proxy, telling the proxy the SID, and which Vera device+serviceId+action to call when it gets an event notification.[/li]
[li]Time passes…[/li]
[li]The device decides to notify subscribers of an event, and one of the subscribers is the proxy on port 2529, so the proxy gets a UPnP NOTIFY event.[/li]
[li]The proxy parses the NOTIFY message and pulls out the SID, the variable and the value.[/li]
[li]The proxy tells the Vera plugin of the event using HTTP lu_action to port 3480, using the action registered in step 3.[/li]
[li]The plugin runs the nominated action.[/li][/ol]

I went through the upnp-event-proxy.lua code and all this is quite clear for me now. 8)

For the step 2, @guessed has already privided the code to subscribe to a Sonos event. 8) I just need to know what is the proxy URL to provide.
Edit: found in your WeMo plugin. I will have to adjust slighty the subscribe function in Sonos plugin to match your subscribeToDevice function.

So, normally, I should be able to test that, as soon as I know how to start your proxy :wink:

Later on, the plugin will need to renew the subscription by repeating the steps, since UPnP subscriptions have built-in obsolescence. The proxy doesn't distinguish initial subscriptions from renewals, it just passes them on to the plugin the same.

It must be renewed at which frequency ?
It means we still need a timer in the plugin to manage this renew :frowning: Isn’t it something that could be managed automatically by the proxy ?

#13

There’s a standalone Lua binary:

lua upnp-event.proxy.lua

It must be renewed at which frequency ? It means we still need a timer in the plugin to manage this renew :( Isn't it something that could be managed automatically by the proxy ?

When the plugin subscribes it suggests a duration for the subscription, and the device responds with either the same or an amended duration. (Search the WeMo source for “Seconds” with a capital S.) The subscription has to be renewed before this time runs out. I’m aiming to have renewals every half an hour, but this is up to each individual plugin.

Yes, it means timers in the plugin. For WeMo, that’s no big deal, because I have to poll the network every so often anyway (and you should too, because even if you have a static DHCP reservation on the UPnP device, UPnP port choice can change if the device reboots).

Some of the design reasons why I am putting the responsibility to renew onto the plugin:

  • The subscription code is already necessarily in the plugin, so it felt messy to duplicate it in the proxy.
  • The proxy can forward notifications to other Veras at different IP addresses, and device UPnP implementations may balk at receiving a renewal from a different IP address than the original subscription.
  • It keeps the proxy lightweight and responsive, since it’s single-threaded. If it was sending out too many packets then you’d experience delays in receiving events too.
  • The plugin may have legitimate reasons not to renew in some circumstances, for reasons not known to the proxy.
#14

Wonderful, I was able to make it work 8) I am able to be notified of volume change. 8)

#15

I might have found a problem.
Now I call the proxy twice because I subscribe to 2 events.
Each time my second call to http.request returns nil as first parameter and “timeout” as second parameter.
Note that the proxy has correctly registered the 2 subscriptions even if the return is wrong, because I see later that I receive notifications from the 2 events.
I tried to increase the timeout in the sock function used in the http.request but it does not help.
I try to put a luup.sleep(10000) between the two calls but it does not help.

I hope I am clear in my explanation.

#16

Like you, I asked 3600 seconds in the SUBSCRIBE message and I got a duration of 3600 seconds as return.
So not a problem to trigger a renewal once per hour.

I imagine we should renew a little before it expires ?

#17

Yes. Often in these protocols the custom for “a little before” is half the expiry period. Paranoid, but easy to calculate.

#18

No idea for my problem ? (returns nill, “timeout”)

#19

Patience, it’s the middle of the workday and I’m trying to type my responses on a tiny phone keyboard.

Two successive PUT calls should work. I admit that I haven’t tested that thoroughly because WeMo has only one interesting service, so only one thing to subscribe to.

Perhaps sockets aren’t getting closed properly. Netstat will help there.

Perhaps there is a deadlock where the plugin is trying to talk to the proxy and the proxy to the plugin, and one of them is timing out.

See if the proxy itself is printing anything interesting, it has its own log.

I’m afraid that you’ll have to experiment.

#20

Here is my proper code … but still the same issue:

[code] local function subscribeToUPnPEvent(subscribeFonct, device, proxyRequestBody)

local sock = function()
	local s = socket.tcp()
	s:settimeout(5)
	return s
end

-- Ask the device to inform the proxy about status changes.
local expiry = nil
local sid, duration = subscribeFonct("http://" .. VERA_IP .. ":2529/upnp/event")
if (sid) then
	expiry = os.time() + duration

	-- Tell proxy about this subscription and the variable we care about.
	-- Volume is the variable we care about.
	proxyRequestBody = proxyRequestBody:format(expiry, device)
	local request, code = http.request({
		url = "http://localhost:2529/upnp/event/" .. url.escape(sid),
		create = sock,
		method = "PUT",
		headers = {
			["Content-Type"] = "text/html",
			["Content-Length"] = proxyRequestBody:len(),
		},
		source = ltn12.source.string(proxyRequestBody),
		sink = ltn12.sink.null(),
	})
	if (request == nil and code ~= "closed") then
		log("Failed to notify proxy of subscription (nil): " .. code, 2)
		sid = nil
		expiry = nil
	elseif (code ~= 200) then
		log("Failed to notify proxy of subscription: " .. code, 2)
		sid = nil
		expiry = nil
	else
		log("Successfully notified proxy of subscription", 2)
	end
end

return sid, expiry

end

function deferredSonosStartup(device)
debug("deferredSonosStartup: start " … device)
device = tonumber(device)

-- Check that the proxy is running.
local sock = function()
	local s = socket.tcp()
	s:settimeout(2)
	return s
end

local t = {}
local request, code = http.request({
	url = "http://localhost:2529/version",
	create = sock,
	sink = ltn12.sink.table(t)
})

local ProxyApiVersion
if (request == nil and code ~= "closed") then
	-- Proxy not running.
	log("Cannot contact UPnP event proxy: " .. code, 2)
	ProxyApiVersion = nil
else
	-- Proxy is running, note its version number.
	ProxyApiVersion = table.concat(t)
end

ip = {}
if (luup.devices[device].device_type == SONOS_DEVICE_TYPE) then
    ip[device] = luup.devices[device].ip or ""
    PARENT_DEVICE = nil
else
    for k, v in pairs(luup.devices)
    do
        if (v.device_num_parent == device and v.device_type == SONOS_DEVICE_TYPE) then
            ip[k] = luup.devices[device].ip or ""
        end
    end
end

for k, v in pairs(ip)
do
    device = k
    local ipAddress = v
    AVTransport[device] = upnp.service(ipAddress, ipPort, UPNP_AVTRANSPORT)
    AudioInput[device] = upnp.service(ipAddress, ipPort, SONOS_AUDIOINPUT)
    Rendering[device] = upnp.service(ipAddress, ipPort, UPNP_RENDERING)
    DeviceProperties[device] = upnp.service(ipAddress, ipPort, UPNP_DEVICE_PROPERTIES)
    ContentDirectory[device] = upnp.service(ipAddress, ipPort, SONOS_MR_CONTENT_DIRECTORY)
    MusicServices[device] = upnp.service(ipAddress, ipPort, SONOS_MUSICSERVICES)

	-- Since Proxy API version 1: accepts NOTIFY from device.
	if (ProxyApiVersion and tonumber(ProxyApiVersion:match("^(%d+)")) >= 1) then
		local sid, expiry = subscribeToUPnPEvent(AVTransport[device].subscribe, device, UPNP_AVTRANSPORT.proxyRequest)
		if (sid ~= nil and expiry ~= nil) then
			log("AVTransport subscription SID " .. sid .. " expiry " .. expiry)
			AVTransport[device].sid = sid
			AVTransport[device].expiry = expiry
		end
		sid, expiry = subscribeToUPnPEvent(Rendering[device].subscribe, device, UPNP_RENDERING.proxyRequest)
		if (sid ~= nil and expiry ~= nil) then
			log("Rendering subscription SID " .. sid .. " expiry " .. expiry)
			Rendering[device].sid = sid
			Rendering[device].expiry = expiry
		end
	end

    sayQueue[device] = {}
    browseAI[device] = true
    queueUri[device] = ""
    metaDataKeys[device] = loadServicesMetaDataKeys(device)
    refreshFirstCache(device)
    refreshSecondCache(device)
end

end[/code]

After calling http.request, is there something like http.close to call ?