Urban Weather Station Netatmo LUA Script for Vera

Screamhouse · August 6, 2013, 11:45am

You can’t put it into the open rain

akbooer · August 7, 2013, 10:47am

I got one of these yesterday and it has already set the gold standard for ease of installation and operation (using its own website and iOS app).

However, taking a look at the excellent documentation (in french) for the Vera script, there seems to be a little room for improvement - in particular, I don’t think it’s using HTTPS (although the URL says that) because it calls http.request and not https.request? It also violates the suggestion that the user credentials are not stored in the app and doesn’t therefore use the access token / refresh token authentication in quite the intended way.

I see the requests below for a proper plugin not using a VContainer… whilst some have asked for separate child devices for the different measurements, there’s no reason not to hang all of the different sensor services/measurements from one device. Does anyone know if there is a proper UPNP category for CO2 and Noise level sensors?

a-lurker · August 7, 2013, 9:57pm

Do I smell a new plugin in the air?

If so - avoiding separate child devices all over the place - IMHO is a better way to go ie if possible, keep all the results consolidated in one place, like the variable container.

Does the UPnP spec mention anything re CO2. In these times, one would think, it may hold some import.

garrettwp · August 8, 2013, 1:32am

By implementing separate child devices with the appropriate device / service id’s would provide better compatibility with 3rd party apps.

Garrett

capjay · August 8, 2013, 4:10am

I agree. However I found that using the VContainer is not a bad idea, it’s supported by the major 3rd party apps, and keeps things tidy. It works with the Data Mine plugin, and can act as a trigger via PLEG.

akbooer · August 8, 2013, 11:00am

Well possibly, but I’m not a fan of JSON or, indeed, Vera UI5 as an interface. I just do everything I can with Lua code loaded as modules at startup, and view results with DataMine and HomeWave.

If so - [u]avoiding[/u] separate child devices all over the place - IMHO is a better way to go ie if possible, keep all the results consolidated in one place, like the variable container.

I agree.

…so therein lies the design conundrum. I suppose that a plugin with options to do either is the ultimate. The problem with VContainer is that although apps like HomeWave can access the data, it’s not presented in the appropriate format for each sensor type.

Does the UPnP spec mention anything re CO2. In these times, one would think, it may hold some import. http://upnp.org/sdcps-and-certification/standards/

I don't know - that's why I asked, same for noise levels - I know where to look but the sheer volume and turgidness of UPnP documentation means I fall asleep before learning anything useful.

system · August 16, 2013, 10:28am

Having trouble with v3 of the netatmo lua file,vcontainer variables do not update. Have run the code on test luup and get the following error:- 01 08/16/13 11:20:58.414 LuaInterface::StartEngine failed run: 0 /usr/lib/lua/json.lua:366: Unexpected character at Line 1 character 307: ] (93) when reading array ({ or [ or ’ or " or number or boolean or null expected)
Context:
_number":7}],“personnalized”:[]},“battery_rint”:0,"battery_vp
^ <0x2fbf5680>
01 08/16/13 11:20:58.414 JobHandler_LuaUPnP::RunLua failed: ------------------------------------------------------------------------

have changed the json.lua file with various versions to no avail. Any help appreciated as I prefer the Netatmo figures for use with PLEG

David

akbooer · August 16, 2013, 11:21am

V3 of the file? Can you be more explicit (where did it come from?)
The only Lua code I have seen is:
[tt]
– AUTHOR: Sebastien Joly
– DATE : 29/01/2013[/tt]

What you have certainly looks like a json parsing error - which json.lua are you, in fact, using?
Which HTTP POST request is generating this error (presumably “getmeasure” ?)

system · August 16, 2013, 1:37pm

I’m using a-lurkers netatmo v3.lua based on sebastion joly’s original code. The json file came with the Google Calendar plugin. Not sure which HTTP Post is generating the error.

capjay · August 16, 2013, 1:47pm

are you sure that your username and password are set correctly, and that they are url-encoded (for example, the ‘@’ character should be replaced with ‘%40’ and so on)

system · August 16, 2013, 2:10pm

are you sure that your username and password are set correctly, and that they are url-encoded (for example, the ‘@’ character should be replaced with ‘%40’ and so on)[/quote]
Yes they are correctly set, and the @ character has been replaced by %40. I’ve been following this thread since I decided to purchase the netatmo station and it has been very helpful.

David

system · August 16, 2013, 4:17pm

I just hacked in my third module and converted to F from C:
seems to work…

------------------------------------------------------------------------
-- NAME:    netatmo-vera.lua
-- AUTHOR:  Sebastien Joly
-- DATE:    29/01/2013
-- COMMENT: Get the data from a Netatmo station
-- Requires the use of two Vera VContainer plugins and a JSON library

------------------------------------------------------------------------
-- Change these values to match the authentication used for your Netatmo account
-- Make sure the values are url encoded. For example: the '@' in an email address if used
-- See http://meyerweb.com/eric/tools/dencoder/
-- You will also need to put in the VContainer IDs further below
------------------------------------------------------------------------
local s_client_id     = "xxxx"
local s_client_secret = "xxxx"
local s_username      = "xxxx"
local s_password      = "xxxxx"


------------------------------------------------------------------------
local request_body = "grant_type=password&client_id=" .. s_client_id .."&client_secret=" .. s_client_secret .. "&username=" .. s_username .. "&password=" .. s_password

------------------------------------------------------------------------
-- Libraries
------------------------------------------------------------------------
require 'ltn12'
require 'socket.http'

-- JSON is not native to Vera. You will need to install a plugin
-- that also installs a json library as a side effect / byproduct
-- Suggest the Google Calendar plugin be used to do this
-- json = require("json") 
json = require("json") 

------------------------------------------------------------------------
-- HTTP POST function
-- refer to: http://w3.impa.br/~diego/software/luasocket/http.html
------------------------------------------------------------------------
function posthttp(p_url,p_body)
    local response_body = { }
    local res, code, response_headers = socket.http.request
    {
      url     = p_url,
      method  = "POST",
      headers =
        {
            ["Content-Type"]   = "application/x-www-form-urlencoded",
            ["Content-Length"] = #p_body
        },
        source = ltn12.source.string(p_body),
        sink   = ltn12.sink.table(response_body)
    }
    return json.decode(table.concat(response_body),true)
end

------------------------------------------------------------------------
-- Main procedure
------------------------------------------------------------------------
local reponsepost  = posthttp("https://api.netatmo.net/oauth2/token",request_body)
local access_token = reponsepost.access_token
reponsepost  = posthttp("https://api.netatmo.net/api/devicelist","access_token=" .. access_token)

local moduleInside  = reponsepost.body.devices[1]._id
local moduleOutside = reponsepost.body.modules[1]._id
local moduleInside2 = reponsepost.body.modules[2]._id

reponsepost = posthttp("https://api.netatmo.net/api/getmeasure","access_token=" ..access_token .."&device_id=" .. moduleInside .. "&scale=max&type=Temperature,CO2,Humidity,Pressure,Noise&date_end=last")
luup.log ("---------------")

local temperatureInside = reponsepost.body[1].value[1][1]
local co2               = reponsepost.body[1].value[1][2]
local humidityInside    = reponsepost.body[1].value[1][3]
local pressure          = reponsepost.body[1].value[1][4]
local noiseLevel        = reponsepost.body[1].value[1][5]

luup.log (temperatureInside)
luup.log (co2)
luup.log (humidityInside)
luup.log (pressure)
luup.log (noiseLevel)



reponsepost = posthttp("https://api.netatmo.net/api/getmeasure","access_token=" ..access_token .."&device_id=" .. moduleInside .. "&module_id=" .. moduleInside2 .. "&scale=max&type=Temperature,Humidity,CO2,Pressure&date_end=last")
local temperatureInside2 = reponsepost.body[1].value[1][1]
local humidityInside2    = reponsepost.body[1].value[1][2]
local co22               = reponsepost.body[1].value[1][3]
local pressure2          = reponsepost.body[1].value[1][4]

luup.log (temperatureInside2)
luup.log (humidityInside2)
luup.log(co22)
luup.log(pressure2)
luup.log ("---------------")








reponsepost = posthttp("https://api.netatmo.net/api/getmeasure","access_token=" ..access_token .."&device_id=" .. moduleInside .. "&module_id=" .. moduleOutside .. "&scale=max&type=Temperature,Humidity&date_end=last")
local temperatureOutside = reponsepost.body[1].value[1][1]
local humidityOutside    = reponsepost.body[1].value[1][2]

luup.log (temperatureOutside2)
luup.log (humidityOutside2)
luup.log ("---------------")













------------------------------------------------------------------------
-- Put the local variables into two VCONTAINER plugins
-- Change the device two IDs to match yours
------------------------------------------------------------------------
local INSIDE_MODULE_ID  = 209 -- VContainer ID: inside
local OUTSIDE_MODULE_ID = 211 -- VContainer ID: outside
local INSIDE_MODULE_ID2 = 212 -- VContainer ID: outside

local VC_SID = "urn:upnp-org:serviceId:VContainer1"

local currentTime = os.date("*t")
currentTime = currentTime.hour .. ":" .. currentTime.min
luup.variable_set(VC_SID, "VariableName3", "Hour", OUTSIDE_MODULE_ID)
luup.variable_set(VC_SID, "Variable3", currentTime, OUTSIDE_MODULE_ID)



-- tempf = ((tempc/5)*9)+32
-- temperature outside
luup.variable_set(VC_SID, "VariableName1", "Temperature F", OUTSIDE_MODULE_ID)
luup.variable_set(VC_SID, "Variable1", ((temperatureOutside/5)*9)+32, OUTSIDE_MODULE_ID)

-- humidity outside
luup.variable_set(VC_SID, "VariableName2", "Humidity", OUTSIDE_MODULE_ID)
luup.variable_set(VC_SID, "Variable2", humidityOutside, OUTSIDE_MODULE_ID)

-- temperature inside
luup.variable_set(VC_SID, "VariableName1", "Temperature F", INSIDE_MODULE_ID)
luup.variable_set(VC_SID, "Variable1", ((temperatureInside/5)*9)+32, INSIDE_MODULE_ID)

-- humidity inside
luup.variable_set(VC_SID, "VariableName2", "Humidity %", INSIDE_MODULE_ID)
luup.variable_set(VC_SID, "Variable2", humidityInside, INSIDE_MODULE_ID)

-- pressure
luup.variable_set(VC_SID, "VariableName3", "Pressure hPa", INSIDE_MODULE_ID)
luup.variable_set(VC_SID, "Variable3", pressure , INSIDE_MODULE_ID)

-- CO2
luup.variable_set(VC_SID, "VariableName4", "CO2 ppm", INSIDE_MODULE_ID)
luup.variable_set(VC_SID, "Variable4", co2, INSIDE_MODULE_ID)

-- noise level
luup.variable_set(VC_SID, "VariableName5", "Noise Level dB", INSIDE_MODULE_ID)
luup.variable_set(VC_SID, "Variable5", noiseLevel, INSIDE_MODULE_ID)



--Now the downstairs one...
-- temperature inside
luup.variable_set(VC_SID, "VariableName1", "Temperature F", INSIDE_MODULE_ID2)
luup.variable_set(VC_SID, "Variable1", ((temperatureInside2/5)*9)+32, INSIDE_MODULE_ID2)

-- humidity inside
luup.variable_set(VC_SID, "VariableName2", "Humidity %", INSIDE_MODULE_ID2)
luup.variable_set(VC_SID, "Variable2", humidityInside2, INSIDE_MODULE_ID2)

-- pressure
luup.variable_set(VC_SID, "VariableName3", "Pressure hPa", INSIDE_MODULE_ID2)
luup.variable_set(VC_SID, "Variable3", pressure2 , INSIDE_MODULE_ID2)

-- CO2
luup.variable_set(VC_SID, "VariableName4", "CO2 ppm", INSIDE_MODULE_ID2)
luup.variable_set(VC_SID, "Variable4", co22, INSIDE_MODULE_ID2)










return true

------------------------------------------------------------------------
-- End program
------------------------------------------------------------------------

akbooer · August 20, 2013, 7:30pm

OK. So here’s a beta version of a Netatmo plugin with these features:

mirrors configuration of any number of stations and modules as assigned to your Netatmo account
creates one ‘master’ plugin device with all the measurements available as device variables
creates child devices for all temperature and humidity sensors
writes a plain-text file of latest measurements to /www/Netatmo.txt which can be read with any browser
uses HTTPS protocol and handles user authentication and access token rotation correctly[/li][/ul]

A few more specifics:

You should ensure that ALL Netatmo modules in your system have unique names (by default, the basic set are “Indoor” and “Outdoor”, but if you have two systems you will have two of each) because…
Luup Netatmo device variables are named from the concatenation of “module” name and “sensor” name, so with a basic configuration you have variables like OutdoorTemperature and IndoorCO2.
Similarly, all the child devices for temperature and humidity are named as “Outdoor - Temperature”, Indoor - Humidity, etc.
On the master plugin device, all the variables are stored under the service urn:akbooer-com:serviceId:Netatmo1
Temperature and Humidity values are stored under their appropriate variable and service names on the child devices, but I’ve been unable to find services for CO2, Pressure, and Noise.

I am a self-confessed JSON-phobe, so have done as little work as possible on the UI side of things. Alas, this means that the plugin has only a generic icon. I have the appropriate .png file, and if someone wants to write the static json file to display it, then fine. All the configuration variables required are accessed under the Advanced tab of the master device.

Installation and Configuration:

Download the three attached files
Create a new device setting the device field to D_Netatmo.xml and description to Netatmo, or some such name
Restart twice

You should now have a startup error for the Netatmo plugin with ‘Authorisation failure’. Critical fields to be filled in under the Advanced tab are:

ClientSecret
ClientID
Username
Password

You already have Username and Password for your Netatmo account. In order to get keys for ClientSecret and ClientID, you need to register your application at http://dev.netatmo.com/dev/createapp

The final required inputs, set to sensible defaults, are:

TokenRefresh - time (in minutes) after which to refresh the access token for the Netatmo web API. These currently expire after 180 minutes, so the default is 120.
MeasurementPoll - rate (in minutes) at which to poll for new measurements. The native Netatmo rate is 5 minutes, so that’s the default. No point in setting to less, and the web API limits the access rate anyway.

I would be interested, of course, in any feedback on this plugin before making it properly available. I haven’t tested it with the optional additional module, or with multiple stations, but if anyone wants to buy me these, I’d be happy to try them. For those who still love the VContainer concept, it’s easy to write a scheduled scene which transfers the measurements you want from the master device to the VContainer for display (I have one running). I’ve tried to meet many of the requirements described earlier in this thread but, of course, in so doing have no doubt compromised the experience for many. Let’s see.

—[Edit]—

I should have said that this requires a json module (inevitably). I’ve used ‘json-dm’ because I already have dataMine installed (who wouldn’t?)

korttoma · August 21, 2013, 4:41am

Greate work akbooer!!

As soon as they got the wind and rain sensors out I will be getting myself some Netatmo for sure.

Thanks for the plugin

Best regards,
Tomas

system · August 21, 2013, 10:09am

Great Plugin Akbooer! I prefer the netatmo sensors even over the Oregon ones.

David

akbooer · August 21, 2013, 4:59pm

I see you have an additional module - can you confirm this beta plugin works correctly? - or send me your additional module to try

— [Edit] —

Actually, a screen shot of the plugin and the six child devices would be compelling…

system · August 21, 2013, 7:01pm

Akbooer

Can confirm beta plugin works with additional netatmo module, so far no bugs have emerged. Hopefully I have attached a screenshot showing the plugin and the 6 child devices!! (They are spread amongst my rooms)

David

akbooer · August 21, 2013, 8:18pm

Excellent news… let’s hope it stays that way!
Impressive setup you have there.

Thanks for the feedback - let me know if you have ideas for improvements. I have a list already, but mostly it’s minor details.

system · August 21, 2013, 9:17pm

installed the device akbooer and it found all 3 netatmo sensors
Perhaps I didn’t see it, but it would be nice to have a Fahrenheit or Celsius toggle.
My temps are showing in Celsius and, as an American, it confuses me

Any luck on finding CO2, Pressure (perhaps choice for atmospheres vs. millibars?) and Noise?

Looks good though!

akbooer · August 21, 2013, 9:39pm

Glad it works for you too with the additional sensor.

Perhaps I didn't see it, but it would be nice to have a Fahrenheit or Celsius toggle. My temps are showing in Celsius and, as an American, it confuses me

The plugin is totally units agnostic, but this can certainly be added. I am away for the next few days but will get back to this.

Any luck on finding CO[sub]2[/sub], Pressure (perhaps choice for atmospheres vs. millibars?) and Noise?

Not yet. This is part of the reason that all the module measurements are reported as variables in the master device. Easy enough (if you program) to pull these out and do what you will with them in a scene.

Looks good though!

Thanks very much for the feedback. Hopefully beta #2 will deliver what you need!