Luasocket's http.request sends the headers all in lower case

I have being trying to connect to a web server using POST using the Lua socket lib. The server accepts the POSTed username and password as a login. I could not get the LUA code to work, while Curl worked perfectly.

I’ve spent “some time” on this problem. It turns out that Luasocket’s http.request sends the headers out all in lower case. This is allowed by the spec. The spec for the headers can be found here:

http://www.ietf.org/rfc/rfc2616.txt

“…Each header field consists of a name followed by a colon (”:“) and the field value. Field names are case-insensitive.”

So it appears the server embedded in the target device is not handling the received headers correctly. The manufacturer of the target device may well fix this in the years to come, in the meanwhile that’s a problem.

The LUA code that lowercases the headers appears to be this LUA code in the function “adjustheaders”:

http://luaxyssl.googlecode.com/svn-history/r67/trunk/lua/socket/http.lua

So the question is and having not used LUA before - how can I override this function with one of my own, that outputs the headers in camel case or whatever case is required by my targeted server?

Further discussion of the problem can be found here:

http://lua.2524044.n2.nabble.com/Tip-for-using-a-Luasocket-s-http-require-for-cranky-webservers-td7630293.html

Authentication problems may be common using the Lua socket lib. Has any one else experienced this issue before?

Ouch, that’s a really unfortunate situation. It’s a bit disappointing that the LuaSocket author is pointing to the spec and suggesting that everybody conform to it: that’s just not realistic.

If you’re only looking for a fix on your own Vera, and don’t need to distribute a patch that will install cleanly from (say) apps.mios.com, then your easiest bet is to simply edit the file at /usr/lib/lua/socket/http.lua. It’s unlikely that you’ll be bitten by your patch’s side-effect (two headers differing only by case) in real life.

I see that the function that you want to replace is lexically scoped (local), so I don’t think that you can just replace socket.http.adjustheaders from the outside:

[code]-- This probably won’t work.
http = require(“socket.http”)

http.adjustheaders = function ()
– … your replacement here …
end
[/code]

I suppose you could put an entire modified copy of the socket library (seven files) into a different directory, and then prepend that copy’s directory into package.path so that it’s found first:

[code]package.path = “/my/patched/luasocket/dir/?.lua;” … package.path

http = require(“socket.http”)
[/code]

I haven’t tested any of this, and things might be confounded with the fact that bits of LuaSocket are compiled. That’s all I know.

As it often happens a 5 minute job turned into one that was a lot, lot longer.

@ futzle - thanks for the info. I had a look at overriding the function without much success, so I modified the underlying http.lua file instead. It’s probably possible to perform the override but I haven’t figured it out. If any one else can demonstrate how the two routines below can be overridden please let us know.

I modified http.lua and have now succeeded in logging into my target device. Turns out that the target required the header to be just so: “Content-Type”. http.lua sent it as “content-type”, resulting in a failed login. Clearly the target web server needs a bit of rework, especially as a lower case “content-length” was of no concern to it.

The basic functionality of the modified http.lua is the same (in theory!) as before. However any user supplied headers now maintain the alpha case as set by the user, instead of being forced to lower case by the code.

Here’s an example showing “any case” headers:

local r, c, h = http.request { url = 'http://some-ip-address/xyz/login.php', method = 'POST', headers = { ["CoNtEnT-TyPe"] = "application/x-www-form-urlencoded", ["cONTENT-lengtH"] = string.len(request_body) }, source = ltn12.source.string(request_body), sink = ltn12.sink.table(response_body) }

Only these two functions, as reproduced below, were changed:

metat.__index:sendbody
adjustheaders

[code]function metat.__index:sendbody(headers, source, step)
source = source or ltn12.source.empty()
step = step or ltn12.pump.step
– if we don’t know the size in advance, send chunked and hope for the best
local mode = “http-chunked”
if headers then
for i,v in base.pairs(headers) do
if (string.lower(i) == “content-length”) then mode = “keep-open” end
end
end
return self.try(ltn12.pump.all(source, socket.sink(mode, self.c), step))
end

local function adjustheaders(reqt)
– duplicate and also lower case the user’s headers
local newHeaders = {}
local lcHeaders = {}
– a simple request with no body, ie a url only, does not use headers
if reqt.headers then
for i,v in base.pairs(reqt.headers) do
newHeaders[i] = v
lcHeaders[string.lower(i)] = v
end
end
– insert the default headers if they are not being provided by the user
if not lcHeaders[“user-agent”] then newHeaders[“user-agent”] = USERAGENT end
if not lcHeaders[“host”] then newHeaders[“host”] = reqt.host end
if not lcHeaders[“connection”] then newHeaders[“connection”] = “close, TE” end
if not lcHeaders[“te”] then newHeaders[“te”] = “trailers” end
– If “authorization” exists the user has supplied it. The header can be in any
– alpha case the user prefers eg: upper, lower, title case, etc but they must
– supply the header value themselves. We just pass on the header untouched.
if not lcHeaders[“authorization”]
then – header was not supplied by the user. If authorization is specified
– only as part of the supplied url, we complete the header here
if reqt.user and reqt.password then
newHeaders[“authorization”] = "Basic " … (mime.b64(reqt.user … “:” … reqt.password))
end
end
return newHeaders
end[/code]

So it looks like the code can be modified without too much drama and give the user more flexibility at the same time.

EDIT: have attached the modified http.lua file to replace /usr/lib/lua/socket/http.lua

Just a bump as this very well may be what’s plaguing me as well. I have an application that I’m interfacing with and I for the life of me can’t get past getting a HTTP Status 501. Everything outside of Vera itself yields successful (200). So I was just about to give up and I found this searching for ‘Lua Socket’. I will definitely have to give this a try.

Yes, a simple job turns into hours of debugging and troubleshooting. If this doesn’t work I’m taking Vera for a long drive into the woods !

How it is possible to replace /usr/lib/lua/socket/http.lua ?
Need to do this to put a new http.lua but I just don’t know how to do it.

Thanks

@bergy1080 You need to learn how to SSH into Vera, so you access its files. See:

http://forum.micasaverde.com/index.php/topic,26995.msg192337.html#msg192337
http://forum.micasaverde.com/index.php/topic,16823.msg235355.html#msg235355

Once you have access to Vera, its just a matter of renaming the old file “http.lua” to say “http.lua.original” and copying the “newhttp.lua” from this thread to same location.