Ok, I added some logs and I can see that my first http.request to the proxy responds after less than 25 ms. The second one fails in timeout every time after 5 s. What is strange is that the timeout I put in the sock function is not taken into account (I tried with s:settimeout(20) for example.
So I think the timeout is too small but its adjustment is not working.
I modified the log function in the UPnP proxy to get timings. So I finally discovered that my second call is handled by the proxy more than 5 s after my call. So it explains the timeout. If I comment the call to “processNotificationQueue” in the main loop, it is working well. As a conclusion, the problem is that the proxy needs a lot of time (several seconds) to handle my first request. Is it normal in your opinion ?
But the main problem is, I think, not in the proxy but in the plugin. How to adjust the timeout when calling http.request in order to have a timeout different from 5 seconds. I would like to set it to 10 s for example. The way you manage it with the sock function seems to not work.
To adjust the http.request timeout, it looks like you have to set http.TIMEOUT first. 5s seems to be the default value.
So I tried to set it to 10 and then 20 but unfortunately it does not fix my problem. It just delayed the return but I still exit in timeout after the first call.
I think I could have the explanation.
As soon as I subscribe to my first sonos event, sonos notifies the proxy and so the sonos plugin while the plugin itself is requesting the proxy for a second subscription and so is blocked.
The problem could be that the proxy cannot notify a plugin when the same plugin is busy to communicate with the proxy.
I will make more tests later to confirm this hypothesis and come back to you.
That’s now perfectly clear for me now. Here is what happens.
In the Sonos plugin, we run a delayed function 1s after the startup function.
luup.call_delay("deferredSonosStartup", 1, lul_device)The function deferredSonosStartup requests several subscriptions to the UPnP proxy.
In the proxy, immediately after the first subscription, the proxy receives a notification from the Sonos device, before the plugin sends its second subscription. The proxy is then blocked on its http.request call to send the notification to the plugin, until the plugin function deferredSonosStartup ends. It finally happens because the next subscription requests end in timeout. Then the proxy can finish its notification sending and continue its job.
That explains why it does not help to increase the timeout on the plugin side.
I finally found the solution. My notification callbacks in the plugin were with tags. I changed to tags and now the proxy does no more block when sending a notification to the plugin 8)
For the renewal, what shall we do exactly ?
In your function subscribeToDevice (WeMo plugin), the function subscribeToDevice has a parameter renewalSID but it is not used in the body of the function. How to pass this parameter in the SUBSCRIBE message ?
When doing the renewal, the SID will not be changed by the UPnP device ?
And after that, do we have to send a new subscription request to the proxy ? It is not necessay, I suppose.
Nice detective work, lolodomo. So it is indeed a deadlock. I think that it might be necessary to add some locking calls to the proxy’s RESTful API to prevent this from happening in the general case. I will have a think about the design. Locking is tricky.
With respect to renewals of subscriptions: the client code in the WeMo plugin for that doesn’t exist yet. The renewalSid variable that you found is the first tendril of that code, and as you can see it doesn’t do anything yet. The intent is that this be the SID of the original subscription, which doesn’t change for the renewal. All you’re really doing is getting back a later expiry time from the UPnP device. After you get the new expiry time back from the UPnP device you’ll need to re-send the PUT command to the proxy to inform it of the new expiry. Currently the proxy stores but doesn’t use the expiry time, but it will use the information in the future to scrub expired subscriptions from its memory, to prevent long-running memory leaks.
Which reminds me of a compromise I thought of that might save you from needing a timer for renewals: the proxy can invoke a lu_action on Vera when a renewal is due, and the Vera plugin can then do the actual renewal call to the UPnP device (or let it lapse).
Thanks for all your work.
Ok but the SID has to be provided in the subscription message. If not, a new SID will be returned. How to do that ?
After you get the new expiry time back from the UPnP device you'll need to re-send the PUT command to the proxy to inform it of the new expiry. Currently the proxy stores but doesn't use the expiry time, but it will use the information in the future to scrub expired subscriptions from its memory, to prevent long-running memory leaks.
Ok
Which reminds me of a compromise I thought of that might save you from needing a timer for renewals: the proxy can invoke a lu_action on Vera when a renewal is due, and the Vera plugin can then do the actual renewal call to the UPnP device (or let it lapse).
Very good idea. Plugin action to invoke could be added in the XML data provided to the proxy during the subscription. And this action would be invoked with the SID as parameter.
Solving this question of renewal is required before releasing any plugin using the proxy, because it is not possible to have a plugin working only during one hour 
Other important point: we have to think how to start the UPnP proxy.
I’ll try and code this up this weekend (including example client code). Thanks for the feedback, it’s really appreciated.
[quote=“lolodomo, post:28, topic:171160”]
Which reminds me of a compromise I thought of …
… which I coded up, and, regrettably, found that it doesn’t work. At some point, when the renewal is due, the proxy was telling the plugin that a renewal was due, and the plugin then renewed with the UPnP device, tried to tell the proxy of the new expiry time, aaaand got a timeout, because the proxy was still waiting for its original reminder to finish. Deadlock.
I thought of workarounds, but they were all just as messy as leaving the plugin to do all the work. So I’ve decided to let the plugin do all the work. Sorry. I haven’t coded this into the WeMo plugin yet, so there’s no sample code yet, sorry. That’s next.
The locking issue that you found earlier in this thread, I have finished coding*, and I’ve put that into the same topic as before. With sample code in the WeMo plugin so you can see how it works. Also I’ve added an extra parameter to the callback, sidParameter, so the plugin can tell if a variable update corresponds with a SID it knows about. If not, it was probably a subscription from before a Luup restart, and the plugin should discard it.
Other important point: we have to think how to start the UPnP proxy.
Indeed. I know how to do this, I’ve done it on generic OpenWrt platforms before. I’ll package it up as a separate app on apps.mios.com, so that you can use it for Sonos, I can use it for WeMo, and whoever else wants to use it can do the same.
Edit: yep, I can think of a deadlock that will still happen. Hold off on amending your code.
Code for the UPnP proxy is now at / – UPnP Event Proxy
Code for the WeMo plugin (client code) is now at / – Belkin WeMo
I’ve taken out the locking API again. I realized that I was using it as a crutch to avoid making the code retry HTTP requests (in both the client and the proxy), and that avoiding retries was futile. Now that I’ve put the timer code into the client it all fits together much more naturally. Renewing a UPnP subscription was practically painless, and the WeMo code now does this, so you can copy it for sample code.
I’ll maintain developer documentation for the proxy at the above URL. Next task is to wrap the proxy into a plugin that can be distributed on apps.mios.com.
Edit: the proxy has been running on my Vera stably for a couple of days. No need for locking, but there is a need for the client plugin to queue up tasks. See the WeMo schedule(), reentry() and queueAction() functions for one way to do this. I’d say that it’s safe to code to the proxy’s current API (3).
Hi futzle.
Now that we have the event notification, what do you advice as a proper way to notify the javascript tab (UI) from the plugin code ?
Currently, my JavaScript tab includes a 1,5 second timer, but I would prefer, if possible, to notify the JavaScript than using this timer.
This is … tricky. I’m not sure that there is a good way. Because the UPnP proxy operates on a different TCP port, the JavaScript on the web interface can’t communicate directly with the proxy. (At least, not without altering the lighttpd configuration file to forward some URLs ? la /port_3481/.)
Sleeping and checking the variable is actually not a bad practice. If you like you can stick it in a loop and retry every half a second, exiting the loop when the status change comes through. That’s what I do in the Caddx plugin, which you can also ransack for sample code.
I do not know of any way to tickle JavaScript code from an Event on the Vera side.
You would have to do a polling AJAX call back to your plugin.
Ok, I will reduce my Javascript timer delay but only refresh when something has changed. That will be at least better than now.
To come back to the UPnP event proxy, I have now installed the version from the app store. No problem, it works very well. Can you confirm that this version logs nothing ? It seems to be confirmed by the reading of your code (output redirected to /dev/null). If I need to (add and) view logs, what is the best way ? Should I kill the process and then start it manually like before with the lua command ? What will happen after a lua reload ? Will your plugin consider my already started process ?
The proxy logs can help to identify the variables to “listen” for example.
Good questions. The current version doesn’t do any proper logging, and what it does is sent to /dev/null in case it interferes with the OpenWrt init process.
For the moment, yes, you could kill the Lua process, run it manually from the command line without redirecting to /dev/null, and then reload Luup. The client plugin will not know the difference.
Longer term, I am thinking that we could log daemon output using [tt]os.execute(“logger %s”)[/tt], passing the log string through the %s; that’d be readable when you run the logread command.
Even longer term, I could have the plugin hack the lighttpd.conf file and add a redirect for port 2529. Then the JavaScript tabs could converse directly with the daemon, get a list of current subscriptions, display them in a table, …
If you feel like doing any of this development, please do. 
I don’t need the JavaScript being notified by the upnp proxy, I needed a way to communicate between the JavaScript tab and my plugin code. Sonos notifies the proxy; the proxy notifies my plugin; and my plugin should be able to notify its ui (JavaScript) . Apparently Micasaverde has not thought about this need.
In the plugin you register your callback with luup.register_handler
From JavaScript … you can poll your Plugin with an asynchronous http request.
You can pass argument/value pairs on the URL for communicating context from JS → LUA.
You can pass a stream of data (in a format appropriate for the two ends) from LUA → JS.
See: http://wiki.micasaverde.com/index.php/Luup_Lua_extensions#function:_register_handler
Hi futzle.
I have a request for you. I see that in the WeMo plugin, you have implemented a mechanism that tries several times to register to the proxy in case the proxy is busy. The problem is that it could happen a certain time between the SUBSCRIBE message and the registration to the proxy, and with Sonos I noticed that the Sonos device immediately notifies after a registration to provide all current data. As a consequence, we could imagine to loose a notification and that would be a big problem in the case of the Sonos.
To be honest, in practice, it seems the proxy is not busy and the first notification arrives just after the proxy registration.
But I think it would be better to have in a unique transaction the event subscription and the proxy registration. Why not adding the event subscription in the proxy code ? It could be an option through additional parameter in the XML data sent to the proxy at registration. If not present, do like now; if present, use them to subscribe first to the event.
If you don’t want to implement this enhancement, I think it would be better for the plugins, when retrying to register to hte proxy, to first renew the event subscription. Doing that, we should avoid any loose of notification (because Sonos sends data after each subscription or renewal of subscription).
Let me know whatr you think about my idea, hoping I was almost clear in my explanations.
Currently, I have not yet handled a proxy busy in my Sonos code, I imagine I should implement a retry mechanism like you did in the WeMo plugin, and if still no chance after few retries, I should fail back to the old plugin behaviour, meaning without the proxy.
Hi lolodomo,
You’ve described a race condition that is inherent to all UPnP control point implementations. Even in a multi-threaded environment like a desktop application, the time between the subscription and the first notification may be too small for the subscribing thread to inform the listening thread of the SID. There’s no way to completely close that window; it’s IMO a weakness in the UPnP spec.
The workaround is for the listening thread (the UPnP event proxy in Vera-speak) to accept notifications for SIDs that it doesn’t yet know about, on the off chance that the subscribing thread (the plugin) hasn’t got around to telling the proxy about the SID. Once the plugin has registered the SID, any notifications from the UPnP device that arrived during that window will be sent back to the plugin. Consequently, events are not lost, but they may be delayed for a second while the proxy retries connecting to the plugin.
The UPnP proxy already implements this “just in case” queuing of notifications for unknown SIDs, so you shouldn’t need to miss any notifications. I tested this a fair amount with the WeMo plugin: WeMo devices are very quick to send their first notification after a subscription has been made.
I’m considering your suggestion to move the responsibility for outgoing subscription requests onto the proxy. The most likely outcome is that it will cause deadlocks in the single-threaded proxy, now that it has to make outgoing connections to machines other than localhost of theoretically unbounded duration. Making the proxy multithreaded (or, more likely, multi-process) seems inevitable. Without reducing the plugin code’s complexity much, I might add: if you are hoping that this will save you from putting retries and delays into your plugin code, then I think you will be disappointed.
Best Home Automation shopping experience. Shop at Ezlo!
© 2024 Ezlo Innovation, All Rights Reserved. Terms of Use | Privacy Policy | Forum Rules