openLuup: AV plugins

[akbooer]

Hallelujah and Happy Thanksgiving! (a bit late) we have identified the problem…

…but not the solution.

The Problem:

You server, quite legitimately, specifies [tt]TRANSFER-ENCODING: chunked[/tt], but does not specify [tt]TRANSFER-LENGTH[/tt], since it also specifies [tt]CONNECTION: close[/tt]. Thus, according to rule 2 of 5 here

@http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4

2.If a Transfer-Encoding header field (section 14.41) is present and has any value other than “identity”, then the transfer-length is defined by use of the “chunked” transfer-coding (section 3.6), unless the message is terminated by closing the connection.

…it should work.

But, the LuaSocket library implementation does not handle this.

The Solution

I need to fix it. When I do, I will feed back to the developers, but it means that I will have to continue to supply and use a modified [tt]http.lua[/tt] module for openLuup. I am quite surprised that this is not handled correctly, but it is, I believe, a fairly obscure use of the protocol (otherwise it would already have been found and fixed.)

[akbooer]

So can you try this as a test of a potential fix…

(stand-alone Lua, as before)

[mda]

[quote=“akbooer, post:102, topic:189395”]So can you try this as a test of a potential fix…

(stand-alone Lua, as before)[/quote]

Thanks, this produces no output. Should i be looking for a log somewhere?

HA-MacBook:~ mda$ lua5.1 /Users/mda/Downloads/http_test.lua HA-MacBook:~ mda$

[akbooer]

Yes, my fault. Wrong line commented out at the end of the file. Can you make sure it’s using the right one and try again.

[mda]

@akbooer

Yes, my fault. Wrong line commented out at the end of the file. Can you make sure it’s using the right one and try again.

Ahh yes, i see it. It now returns:

[code]HA-MacBook:~ mda$ lua5.1 /Users/mda/Downloads/http_test.lua
— status —
HTTP/1.1 200 OK
— headers —
TRANSFER-ENCODING: chunked
CONTENT-TYPE: text/xml
DATE: Sat, 28 Nov 2015 00:02:42 GMT
LAST-MODIFIED: Fri, 20 Nov 2015 14:30:59 GMT
SERVER: Linux/2.6.28.3, UPnP/1.0, Portable SDK for UPnP devices/1.6.6
X-User-Agent: redsonic
CONNECTION: close

— chunk —

<?xml version="1.0"?> nil

lua5.1: /Users/mda/Downloads/http_test.lua:82: attempt to index upvalue ‘headers’ (a nil value)
stack traceback:
[C]: in function ‘request’
/Users/mda/Downloads/http_test.lua:383: in main chunk
[C]: ?[/code]

Just for comparison i ran it against the DLNA renderer that is working in openluup (192.168.1.14:8080/description.xml) and it returns:

HA-MacBook:~ mda$ lua5.1 /Users/mda/Downloads/http_test.lua --- status --- HTTP/1.1 200 OK --- headers --- CONTENT-TYPE: text/xml;charset="utf-8" CONTENT-LENGTH: 3084

[akbooer]

Sigh. I test it on both chunked and non-chunked servers, but I don’t have one that behaves quite as yours does.

Let’s try this again…

[mda]

@akbooer

Let’s try this again…

Success I think?

[code]HA-MacBook:~ mda$ lua5.1 /Users/mda/Downloads/http_test-2.lua
— status —
HTTP/1.1 200 OK
— headers —
TRANSFER-ENCODING: chunked
CONTENT-TYPE: text/xml
DATE: Sat, 28 Nov 2015 17:13:13 GMT
LAST-MODIFIED: Fri, 20 Nov 2015 14:30:59 GMT
SERVER: Linux/2.6.28.3, UPnP/1.0, Portable SDK for UPnP devices/1.6.6
X-User-Agent: redsonic
CONNECTION: close

— chunk —

<?xml version="1.0"?> nil 1 0 urn:schemas-upnp-org:device:MediaRenderer:1 /renderpres.html ROCKI_Sam Ivo Clarysse http://gmrender.nongnu.org/ gmediarender 0.0.6 Epitome http://gmrender.nongnu.org/ uuid:5f9ec1b3-ed59-1900-4530-00117FB7DF48 12 DMR-1.50 image/png 64 64 24 /upnp/grender-64x64.png image/png 128 128 24 /upnp/grender-128x128.png urn:schemas-upnp-org:service:AVTransport:1 urn:upnp-org:serviceId:AVTransport /upnp/rendertransportSCPD.xml /upnp/control/rendertransport1 /upnp/event/rendertransport1 urn:schemas-upnp-org:service:ConnectionManager:1 urn:upnp-org:serviceId:ConnectionManager /upnp/renderconnmgrSCPD.xml /upnp/control/renderconnmgr1 /upnp/event/renderconnmgr1 urn:schemas-upnp-org:service:RenderingControl:1 urn:upnp-org:serviceId:RenderingControl /upnp/rendercontrolSCPD.xml /upnp/control/rendercontrol1 /upnp/event/rendercontrol1 http://192.168.1.125:49153/ nil --- chunk --- nil closed[/code]

[akbooer]

@mda

Success I think?

Well, maybe. Try this as a replacement for openLuup/http.lua, ensuring the openLuup/server.lua does a ‘require’ for this file (I think you changed that a while ago?)

[mda]

@akbooer

[quote=“mda, post:107, topic:189395”]Success I think?

Well, maybe. Try this as a replacement for openLuup/http.lua, ensuring the openLuup/server.lua does a ‘require’ for this file (I think you changed that a while ago?)[/quote]

Didn’t work :(. The Discover button still only found the 1 renderer and when i pasted in the direct link it did not work. Log output is:

2015-11-28 12:24:13.821 openLuup.server:: /data_request?id=lu_action&output_format=xml&DeviceNum=4&serviceId=urn:dlna-org:serviceId:DLNAMediaController1&action=SelectDMRDevice&URL=http%3A%2F%2F192.168.1.125%3A49153%2Fdescription.xml tcp{client}: 0x7ff41c84d828 2015-11-28 12:24:13.821 luup.call_action:0: 4.urn:dlna-org:serviceId:DLNAMediaController1.SelectDMRDevice 2015-11-28 12:24:13.821 luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.DescriptionURL was: http://192.168.1.14:8080/description.xml now: http://192.168.1.125:49153/description.xml #hooks:0 2015-11-28 12:24:13.821 luup_log:4: DLNA: debug: Cancelling all event subscriptions 2015-11-28 12:24:13.821 luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.ModelName was: *AVR-4520 now: #hooks:0 2015-11-28 12:24:13.821 luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.Online was: 1 now: 0 #hooks:0 2015-11-28 12:24:13.821 luup_log:4: DLNA: debug: Switching to a new DMR: setting variable Online to 0 2015-11-28 12:24:13.844 luup_log:4: DLNA: error: UPnP_getDeviceDescription wget failed - status=invalid chunk size xml= 2015-11-28 12:24:13.844 luup_log:4: DLNA: debug: upnp.setup failed: going offline

I tried it with chunking enabled and disabled, the result was he same. I confirmed my server.la file contains:

local http      = require "openLuup.http"

---

I am not sure if this matters but when i run the latest standalone http_test.lua on the renderer openluup does find (.14), i just get:

HA-MacBook:cmh-ludl mda$ lua5.1 /Users/mda/Downloads/http_test-2.lua --- status --- HTTP/1.1 200 OK --- headers --- CONTENT-TYPE: text/xml;charset="utf-8" CONTENT-LENGTH: 3084

When i tun it on the renderer that does not (.125) work, i get this:

[code]HA-MacBook:cmh-ludl mda$ lua5.1 /Users/mda/Downloads/http_test-2.lua
— status —
HTTP/1.1 200 OK
— headers —
TRANSFER-ENCODING: chunked
CONTENT-TYPE: text/xml
DATE: Sat, 28 Nov 2015 20:37:55 GMT
LAST-MODIFIED: Fri, 20 Nov 2015 14:30:59 GMT
SERVER: Linux/2.6.28.3, UPnP/1.0, Portable SDK for UPnP devices/1.6.6
X-User-Agent: redsonic
CONNECTION: close

— chunk —

<?xml version="1.0"?> nil 1 0 urn:schemas-upnp-org:device:MediaRenderer:1 /renderpres.html ROCKI_Sam Ivo Clarysse http://gmrender.nongnu.org/ gmediarender 0.0.6 Epitome http://gmrender.nongnu.org/ uuid:5f9ec1b3-ed59-1900-4530-00117FB7DF48 12 DMR-1.50 image/png 64 64 24 /upnp/grender-64x64.png image/png 128 128 24 /upnp/grender-128x128.png urn:schemas-upnp-org:service:AVTransport:1 urn:upnp-org:serviceId:AVTransport /upnp/rendertransportSCPD.xml /upnp/control/rendertransport1 /upnp/event/rendertransport1 urn:schemas-upnp-org:service:ConnectionManager:1 urn:upnp-org:serviceId:ConnectionManager /upnp/renderconnmgrSCPD.xml /upnp/control/renderconnmgr1 /upnp/event/renderconnmgr1 urn:schemas-upnp-org:service:RenderingControl:1 urn:upnp-org:serviceId:RenderingControl /upnp/rendercontrolSCPD.xml /upnp/control/rendercontrol1 /upnp/event/rendercontrol1 http://192.168.1.125:49153/ nil --- chunk --- nil closed[/code]

[akbooer]

No, this is fine. BTW, chunking for the port 3480 server should have nothing to do with it.

The problem now is that the DLNA code is still using the wrong library. So in both [tt]L_DLNAUPnPUtility.lua[/tt] and [tt]L_DLNATTS.lua[/tt] you need to change the line which says:

local http = require("socket.http")

to this:

local http = require("openLuup.http")

Any better?

---

Edit: Alternatively, substitute the actual [tt]socket.http[/tt] in your Lua install library… although perhaps best wait until we’re sure it’s working!

[mda]

@akbooer

Any better?

No better

2015-11-28 15:23:04.066   openLuup.server:: /data_request?id=lu_action&output_format=xml&DeviceNum=4&serviceId=urn:dlna-org:serviceId:DLNAMediaController1&action=SelectDMRDevice&URL=http%3A%2F%2F192.168.1.125%3A49153%2Fdescription.xml tcp{client}: 0x7fd4e2025828
2015-11-28 15:23:04.066   luup.call_action:0: 4.urn:dlna-org:serviceId:DLNAMediaController1.SelectDMRDevice 
2015-11-28 15:23:04.066   luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.DescriptionURL was: http://192.168.1.14:8080/description.xml now: http://192.168.1.125:49153/description.xml #hooks:0
2015-11-28 15:23:04.068   luup_log:4: DLNA: debug: Cancelling all event subscriptions
2015-11-28 15:23:04.068   luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.ModelName was: *AVR-4520 now:  #hooks:0
2015-11-28 15:23:04.068   luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.Online was: 1 now: 0 #hooks:0
2015-11-28 15:23:04.068   luup_log:4: DLNA: debug: Switching to a new DMR: setting variable Online to 0
2015-11-28 15:23:09.074   luup_log:4: DLNA: error: UPnP_getDeviceDescription wget failed - status=timeout xml=
2015-11-28 15:23:09.074   luup_log:4: DLNA: debug: upnp.setup failed: going offline

L_DLNAUPnPUtility.lua now has:

[code]module(“L_DLNAUPnPUtility”, package.seeall)

local url = require(“socket.url”)
local socket = require(“socket”)
local http = require(“openLuup.http”) – for openluup
– local http = require(“socket.http”)
local ltn12 = require(“ltn12”)[/code]

and L_DLNATTS.lua now has:

[code]module(“L_DLNATTS”, package.seeall)

local url = require(“socket.url”)
local socket = require(“socket”)
– local http = require(“socket.http”)
local http = require(“openLuup.http”) – for openluup
local https = require “ssl.https”
local ltn12 = require(“ltn12”)
[/code]

I reverted server.lua back to using the chunking with just the require change:

[code]local _NAME = “openLuup.server”
local revisionDate = “2015.11.01”
local banner = " version " … revisionDate … " @akbooer"

–
– openLuup SERVER - HTTP GET request server and client

local socket = require “socket”
local url = require “socket.url”
– local http = require “socket.http” – appears to create global variable TIMEOUT
local http = require “openLuup.http”[/code]

Just in case i also tried with the chunking turned off in server.lua but it did not make a difference.

Thanks

[akbooer]

OK. Quick sanity check:

[ul][li]new http.lua file is installed in openLuup/ directory[/li]
[li]openLuup server is using this new file[/li]
[li]so are both of the DLNA .lua files[/li][/ul]

That’s my understanding of the current setup.

I see that the error is now “timeout” whereas it was “invalid chunk size” ? This is progress (?)

Can you clarify one thing: it looked as though the stand-alone test file was working, but what does this do (once again) when run from AltUI’s Lua Test window?

local a,b = luup.inet.wget "http://192.168.1.125:49153/description.xml"
print (a,b)

Meanwhile, I’ll take another, closer, look at all the DLNA code. But we are perhaps not quite done with http.lua either… both pieces of code I have not written myself.

[mda]

[quote=“akbooer, post:112, topic:189395”]OK. Quick sanity check:

[ul][li]new http.lua file is installed in openLuup/ directory[/li]
[li]openLuup server is using this new file[/li]
[li]so are both of the DLNA .lua files[/li][/ul]

That’s my understanding of the current setup.[/quote]

Yes, all correct. Do you perhaps want to give me an updated http.lua that writes something to the log when it is called to confirm it is actually being invoked (and perhaps provides some info that would be useful to you in debugging?)

When i run the test is altui i get:

[code]Return Result:
nil

Console Output:
invalid chunk size [/code]

Thanks.

[akbooer]

Ah, now that I do understand: AltUI is also using the wrong http.lua.

One way around this is, as I mentioned previously, to replace that file in your basic Lua installation. That way, everything will use it. The modifications I’ve made will not have any affect on ‘normal’ HTTP traffic.

Could you, perhaps, just modify the last successful http_test.lua file to print the two return parameters from the request?

[mda]

[quote=“akbooer, post:114, topic:189395”]Ah, now that I do understand: AltUI is also using the wrong http.lua.

One way around this is, as I mentioned previously, to replace that file in your basic Lua installation. That way, everything will use it. The modifications I’ve made will not have any affect on ‘normal’ HTTP traffic.

Could you, perhaps, just modify the last successful http_test.lua file to print the two return parameters from the request?[/quote]

I replaced the http.lua in my directory. It still does not work and now it also does not find the Renderer it used to find when i click the Discover button.

[code]HA-MacBook:cmh-ludl mda$ grep DLNA LuaUPnP.log

2015-11-29 17:06:51.718 luup_log:4: DLNA: debug: refreshNow: start
2015-11-29 17:06:51.718 luup_log:4: DLNA: debug: GetTransportInfo(‘http://192.168.1.14:8080/AVTransport/ctrl’, ‘urn:schemas-upnp-org:service:AVTransport:1’) Called with parameter count=1
2015-11-29 17:06:51.718 luup_log:4: DLNA: debug: UPnP_request: url=[http://192.168.1.14:8080/AVTransport/ctrl], body=[<?xml version="1.0" encoding="utf-8"?>
2015-11-29 17:06:56.718 luup_log:4: DLNA: debug: AVTransport.GetTransportInfo failed: going offline
2015-11-29 17:06:56.718 luup_log:4: DLNA: debug: commsFailure: Device offline? status=timeout
2015-11-29 17:06:56.719 luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.Online was: 1 now: 0 #hooks:0
:80/cmh/skins/default/icons/DLNA.png #hooks:0
:80/cmh/skins/default/icons/DLNA.png #hooks:0
2015-11-29 17:06:56.720 luup.variable_set:4: 4.urn:upnp-org:serviceId:ConnectionManager.SinkProtocolInfo was: http-get::audio/L16;rate=44100;channels=1:DLNA.ORG_PN=LPCM,http-get::audio/L16;rate=44100;channels=2:DLNA.ORG_PN=LPCM,http-get::audio/L16;rate=48000;channels=1:DLNA.ORG_PN=LPCM,http-get::audio/L16;rate=48000;channels=2:DLNA.ORG_PN=LPCM,http-get::audio/mpeg:DLNA.ORG_PN=MP3,http-get::audio/x-ms-wma:DLNA.ORG_PN=WMABASE,http-get::audio/x-ms-wma:DLNA.ORG_PN=WMAFULL,http-get::audio/mp4:DLNA.ORG_PN=AAC_ISO,http-get::audio/3gpp:DLNA.ORG_PN=AAC_ISO,http-wavetunes::audio/x-ms-wma:,http-get::audio/wav:,http-get::audio/x-wav:,http-get::audio/flac:,http-get::audio/x-flac:,http-get::audio/x-m4a:,http-get::audio/x-mp4:,http-get::image/jpeg:* now: #hooks:0
2015-11-29 17:06:56.720 luup_log:4: DLNA: debug: Cancelling all event subscriptions
2015-11-29 17:06:56.720 openLuup.server:: /data_request?id=lu_action&output_format=xml&DeviceNum=4&serviceId=urn:dlna-org:serviceId:DLNAMediaController1&action=StartDMRDiscovery tcp{client}: 0x7fa88285e228
2015-11-29 17:06:56.723 luup.call_action:0: 4.urn:dlna-org:serviceId:DLNAMediaController1.StartDMRDiscovery
2015-11-29 17:06:56.729 luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.DMRDiscoveryResult was: 192.168.1.148080http://192.168.1.14:8080/description.xml*AVR-4520Theater & Whole Home Audio now: scanning #hooks:0
2015-11-29 17:07:02.875 luup_log:4: DLNA: error: UPnP_getDeviceDescription wget failed - status=closed xml=
2015-11-29 17:07:02.950 luup_log:4: DLNA: error: UPnP_getDeviceDescription wget failed - status=closed xml=
2015-11-29 17:07:02.980 luup_log:4: DLNA: error: UPnP_getDeviceDescription wget failed - status=closed xml=
2015-11-29 17:07:02.981 luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.DMRDiscoveryResult was: scanning now: #hooks:0
2015-11-29 17:07:06.876 luup_log:4: DLNA: debug: refreshNow: start
2015-11-29 17:07:06.876 luup_log:4: DLNA: debug: GetTransportInfo(‘http://192.168.1.14:8080/AVTransport/ctrl’, ‘urn:schemas-upnp-org:service:AVTransport:1’) Called with parameter count=1
2015-11-29 17:07:06.876 luup_log:4: DLNA: debug: UPnP_request: url=[http://192.168.1.14:8080/AVTransport/ctrl], body=[<?xml version="1.0" encoding="utf-8"?>
2015-11-29 17:07:11.878 luup_log:4: DLNA: debug: AVTransport.GetTransportInfo failed: going offline
2015-11-29 17:07:11.878 luup_log:4: DLNA: debug: commsFailure: Device offline? status=timeout
2015-11-29 17:07:11.879 openLuup.server:: /data_request?id=lu_action&output_format=xml&DeviceNum=4&serviceId=urn:dlna-org:serviceId:DLNAMediaController1&action=SelectDMRDevice&URL=http%3A%2F%2F192.168.1.125%3A49153%2Fdescription.xml tcp{client}: 0x7fa88285e228
2015-11-29 17:07:11.879 luup.call_action:0: 4.urn:dlna-org:serviceId:DLNAMediaController1.SelectDMRDevice
2015-11-29 17:07:11.879 luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.DescriptionURL was: http://192.168.1.14:8080/description.xml now: http://192.168.1.125:49153/description.xml #hooks:0
2015-11-29 17:07:11.879 luup_log:4: DLNA: debug: Cancelling all event subscriptions
2015-11-29 17:07:11.879 luup.variable_set:4: 4.urn:dlna-org:serviceId:DLNAMediaController1.ModelName was: *AVR-4520 now: #hooks:0
2015-11-29 17:07:11.879 luup_log:4: DLNA: debug: Switching to a new DMR: setting variable Online to 0
2015-11-29 17:07:11.931 luup_log:4: DLNA: error: UPnP_getDeviceDescription wget failed - status=closed xml=
2015-11-29 17:07:11.931 luup_log:4: DLNA: debug: upnp.setup failed: going offline
2015-11-29 17:07:22.196 luup_log:4: DLNA: debug: refreshNow: start[/code]

I would be happy to make the change to log but i am not sure what to do. could you possibly tell me what to add to the file, and where, to get the debugging info that would be useful? Thanks!

[akbooer]

@mda

…it also does not find the Renderer it used to find when i click the Discover button.

That is interesting.

...could you possibly tell me what to add to the file, and where, to get the debugging info that would be useful? Thanks!

Let’s just go back to stand-alone testing. Try this.

[mda]

@akbooer

Let’s just go back to stand-alone testing. Try this.

Thanks. Results are:

[code]HA-MacBook:~ mda$ lua5.1 /Users/mda/Downloads/http_test.lua
— status —
HTTP/1.1 200 OK
— headers —
TRANSFER-ENCODING: chunked
CONTENT-TYPE: text/xml
DATE: Mon, 30 Nov 2015 22:35:04 GMT
LAST-MODIFIED: Fri, 20 Nov 2015 14:30:59 GMT
SERVER: Linux/2.6.28.3, UPnP/1.0, Portable SDK for UPnP devices/1.6.6
X-User-Agent: redsonic
CONNECTION: close

— chunk —

<?xml version="1.0"?> nil 1 0 urn:schemas-upnp-org:device:MediaRenderer:1 /renderpres.html ROCKI_Sam Ivo Clarysse http://gmrender.nongnu.org/ gmediarender 0.0.6 Epitome http://gmrender.nongnu.org/ uuid:5f9ec1b3-ed59-1900-4530-00117FB7DF48 12 DMR-1.50 image/png 64 64 24 /upnp/grender-64x64.png image/png 128 128 24 /upnp/grender-128x128.png urn:schemas-upnp-org:service:AVTransport:1 urn:upnp-org:serviceId:AVTransport /upnp/rendertransportSCPD.xml /upnp/control/rendertransport1 /upnp/event/rendertransport1 urn:schemas-upnp-org:service:ConnectionManager:1 urn:upnp-org:serviceId:ConnectionManager /upnp/renderconnmgrSCPD.xml /upnp/control/renderconnmgr1 /upnp/event/renderconnmgr1 urn:schemas-upnp-org:service:RenderingControl:1 urn:upnp-org:serviceId:RenderingControl /upnp/rendercontrolSCPD.xml /upnp/control/rendercontrol1 /upnp/event/rendercontrol1 http://192.168.1.125:49153/ nil --- chunk --- nil closed --- status --- closed --- result --- nil[/code]

[mda]

@akbooer is there anything more we can try? Would it help if I made one of these ROCKI dlna renders accessible to you over the Internet to play with? Thanks!

Sent from my iPhone using Tapatalk

[akbooer]

@mda

is there anything more we can try?

Yes, most definitely. We have made great progress, although it might not seem so. We can identify the exact cause of the problem. My quick fix for that was not a complete fix. It means that I don’t understand enough about the LuaSocket library, and need to dig deeper.

Would it help if I made one of these ROCKI dlna renders accessible to you over the Internet to play with? Thanks!

Well my next step was to simulate the server response (I think a have enough data to do that) but actually having access would mean I needn’t do that. So yes please.

[mda]

@akbooer

Well my next step was to simulate the server response (I think a have enough data to do that) but actually having access would mean I needn’t do that. So yes please.

Sent you a PM. Thanks !

Also, do you know if the Weather Underground plugin or Ecobee3 thermostat plugin will work? Thanks.