Would incomming and ser2net work just as on Vera?

Hi akbooer,

On the Vera you can attach a serial to USB cable for serial communications. You link that to a plug-in and in the plug-ins incomming function you can capture any data sent over. The Vera configures ser2net for that on port 3481 (well at least on mine, probably bump that number if you have more attached).

Would this work on openLuup? Or with ser2net configured, would you be able to luup.io.open to address 127.0.0.1 and port 3481?

If so I have two more plug-ins I can try to port to openLuup. If not I’m sort of stuck.

Cheers Rene

Yes, this should work, AFAIK. In fact, I’m sure someone has done this already, and posted about it… I just need to find that post. It’s for this reason that I mentioned that there are work-arounds for the lack of dedicated serial interface support in openLuup.

Edit:

@martynwendon appears to have done that for the Visionic alarm panel, here: http://forum.micasaverde.com/index.php/topic,36090.msg270898.html#msg270898

…also here…

Hi akbooer,

Thanks for the reply. I installed it and it sort of works. I am not using a seperate serial to Ethernet device as a PI can do that is my thinking.

I can see data is comming in, but chunks are missing. Not all data is received, the beginning of a line is nearly always missing.

For the dutch Smart meter you get about 20 lines each 10 seconds at 115200 baud, a Veralite can cope with that so I do expect a Pi2 can too.

2016-08-30 19:02:23.229   openLuup.scheduler:: [11] device startup
2016-08-30 19:02:23.229   luup_log:11: Smart Meter: Starting Smart Meter device: 11
2016-08-30 19:02:23.230   luup_log:11: Smart Meter: IP = 192.168.178.21, port = 3481
2016-08-30 19:02:23.231   luup.io.open:11: connecting to 192.168.178.21:3481, using crlf protocol tcp{client}: 0x1e1bb00
2016-08-30 19:02:23.231   openLuup.io:11: connect OK
.....
2016-08-30 18:50:13.852   openLuup.io:11: timeout tcp{client}: 0xe0cf30
2016-08-30 18:50:13.854   luup.io.incoming:11: bytes received: 4, status: OK tcp{client}: 0xe0cf30
2016-08-30 18:50:13.855   luup_log:11: Smart Meter: No key found in : ETER
2016-08-30 18:50:13.856   luup.io.incoming:11: bytes received: 0, status: OK tcp{client}: 0xe0cf30
2016-08-30 18:50:13.882   openLuup.io:11: timeout tcp{client}: 0xe0cf30
2016-08-30 18:50:13.883   luup.io.incoming:11: bytes received: 8, status: OK tcp{client}: 0xe0cf30
2016-08-30 18:50:13.884   luup_log:11: Smart Meter: Not processing : .2.8(42)
2016-08-30 18:50:13.911   openLuup.io:11: timeout tcp{client}: 0xe0cf30
2016-08-30 18:50:13.913   openLuup.io:11: timeout tcp{client}: 0xe0cf30
....

I tried to figure out where the "timeout tcp … message comes from, but I can’t

Hope you can help and give some pointers.

Cheers Rene

The io.lua module has, ostensibly, a 5 second timeout hard wired in the code. Easily changed, but I’ve had a great deal of trouble with timeout handling in the LuaSocket library. For the HTTP server, I think I had to change this to infinite to get things to work as expected.

On the other hand, I’d check carefully without using openLuup that everything was working first at the UNIX shell level using netcat or whatever.

Hi,

I made a very short bit of LUA code that opens the socket as the openluup.io module does and then just reads line by line. That looked to work just fine. I only had to increase the timeout on open to account for the time no data is received between two samples, I thought.

I also increased that time in the openLuup io module, even to 120 seconds. It then only takes longer before the timeouts start to happen, but they keep happening. It even seems the timeout counter is not reset. That is, when setting the timeout to 120 seconds, the timeouts on the socket:receive start to happen 120 seconds after opening it, even when reading data.

This same effect I see in my test code after the timeout seconds on the connect expire you get the timeout. In my test code I set the timeout to nil and then it keeps running.

So I think that in the io.open.connect function you have to do this as well to make sure the port for the incomming handler stays open for as long as LUA is running. I made that change on my system and so far so good.

Taking this load of the Vera and onto my openLuup system should make some difference in reliability :smiley:

Cheers Rene

[quote=“reneboer, post:5, topic:193534”]In my test code I set the timeout to nil and then it keeps running.

So I think that in the io.open.connect function you have to do this as well to make sure the port for the incomming handler stays open for as long as LUA is running. I made that change on my system and so far so good.[/quote]

Interesting. Yes, that is the solution I was mentioning for the HTTP handler. This is odd, because I have some plugins - the MySensors Arduino one, for example, which works just fine with the IO module the way it is, and I have had one of those running for literally months between reloads.

You just set the OPEN_SOCKET_TIMEOUT parameter to nil?

…the only problem with this is that if the remote end does not respond then the whole system hangs…

Hi akbooer,

Yup, just set that to nil. With your socket_callbacks scheduler checking for a socket with data (socket.select) wouldn’t that avoid a hangup? The system keeps working fine, also for the ten seconds nothing is transmitted by the SmartMeter.

And maybe this is an Rapberry Pi Jessie/Debian issue, no idea. I agree that it seems odd the timeout does not seem to care data is received. I did stumble onto this [url=https://github.com/diegonehab/luasocket/issues/77]https://github.com/diegonehab/luasocket/issues/77[/url] and looking at the luasocket source code (V3.0) for my Pi this issue/behaviour still seems to exist. They do mention that socket.select is a way typically used to avoiding locks. They even say power users do. So I guess you are one (I know you are one ;D)

Cheers Rene

Well thanks for the vote of confidence! Yes, socket.select has its uses (and, indeed, IS used in the openLuup scheduler.)

Yes, it’s fine if you’ve got an opened socket which is not sending or receiving data, but the issue is if the initial opening connection does not complete then, in my experience, you can get the system hanging.

But it’s better that when it runs, it runs correctly, rather than not working at all!

Hi akbooer,

I will do some tests with unplugging the cable at different points and see if it gets hung up or not. I will let you know.

I’m just thinking this same behaviour could also be why I get incorrect responses from the Harmony hub on the Pi every so often. Also worth some investigation I guess.

Cheers Rene

Hi,

Little test update. With the timeout at nil the system kept running like a charm. No negative effects…

Until I unplugged the serial cable. Every thing came to a halt, and I mean everything even the GUI (ALTUI) stopped responding. So indeed not such a good approach.

So I am now running a new test. The timeout back to the 5 sec default. In the incoming function I check for a timeout. If that is returned by the receive function then I simply do a connect call again. So far so good. I’ll keep it running for a couple of days and let you know.

  local function incoming (sock)
    local data, err = receive (sock, protocol)        -- get data
    if data then
      _log (("bytes received: %d, status: %s %s"): format ((#data or 0), err or "OK", tostring(sock)), "luup.io.incoming")
      local ok, msg = scheduler.context_switch (devNo, dev.io.incoming, devNo, data) 
      if not ok then _log(msg) end
    else
      if err == "closed" then 
        sock: close ()            -- close our end
        scheduler.socket_unwatch (sock)
        _log ("socket connection closed " .. tostring (sock))
      elseif err == "timeout" then   -- on timeout reopen socket
        local ok, msg = sock:connect (ip, port) 
      else
        _log ((err or '?') .. ' incomming ' .. tostring (sock))
      end
    end
  end

Cheers Rene

Yes,I think that’s the right approach. I was planning to make such a change before using the io module for asynchronous connections for the VeraBridge, which will cut down on processing and network bandwidth. So I’ll be happy for you to get it going first :wink:

I am seeing some data loss after the timeout. Not critical for my current application as the first line is not used, but that needs some more work.

Cheers Rene

I’m just discovering this thread now. I should have checked on the forum before searching a solution for my USB devices.

Hi,

I cannot seem to get the data loss on the first line fixed. I tried to see if the receive command would return any data when there is a timeout, but it does not. I guess that is because it is the CRLF protocol and the receive function waits for the LF character before returning data. To avoid that I think you would have to read character by character (raw), and reassemble in the receive function. But that would add much more processing and complexity.

For now the Smart Meter Reader works with reconnecting after timeout and increasing the timeout to 30 seconds, so I think I wait for winter time to dig a bit deeper.

Cheers Rene