New Plugin: SiteSensor

Plugins & Plugin Development General Plugin Discussion
122

Look at my 5th post back.

123

I saw it. I put that exact response through the expression parser with the expressions I just gave you and it works fine.

124

Let’s go deep. Request the URL below. Modify it with the local IP address of your Vera.

http://your-vera-ip/port_3480/data_request?id=lr_SiteSensor&action=status

PM that to me (don’t reply here), and please put ``` (three back-ticks) on a line by itself both before and after the returned content.

125

When you go into the settings and look at the request URL, does it say &symbol= or &symbol in the middle?

126

OK. Got your PMs. So the corruption of the URL I’m seeing is just a function of Discourse’s formatting. Man, we’ve all got a big learning curve here.

Even so, having corrected the URL, the JSON you sent, you must have gotten from somewhere other than SiteSensor, because SiteSensor has no data whatsoever, in your configuration and in my testing. SiteSensor never got a response from the server. It never even got far enough in the connection process to make the request.

The reason for that is that the connection to the server is crashing at a low level. This is a really weird server. It uses software called gnunicorn, which is pretty uncommon (you generally see more Apache, nginx, and IIS).

Opening the server directly failed on every version of OpenSSL I could find
 on Vera, on CentOS 7, on Ubuntu, many different versions. All produce “alert 80” in the TLS negotiation and fail.

But curl works. So that’s weird. At first I thought curl was working because older curls can be built on SSLEay, a different library, but every version I was using (including Vera’s) was built on OpenSSL (visible in curl -version). So, what is curl doing specially
???

More sleuthing and experimentation revealed that if I set the TLS extension servername for the ClientHello in OpenSSL (how’s that for a mouthful?), the connection succeeds. curl must be setting that to the hostname when it makes the request. LuaSec, apparently, does not.

Unfortunately, that option does not appear to be settable in LuaSec, the interface that Lua uses to OpenSSL. At least, not that I’ve found yet. I may have to go back to the package maintainer on that, but it doesn’t stand out in the documentation. If not, or more importantly, if it is but not in the particular version of LuaSec Vera uses, there will be no fix (at least, not until LuaSec has the ability and Vera upgrades to a version of LuaSec that has the ability).

All is not lost, however. There is an obvious workaround: what I can do without much pain is have SiteSensor just use curl as an alternate conduit for the data. So, I’m testing a version now that just lets you (by configuring in a state variable) use curl rather than LuaSocket/LuaSec.

Stay tuned


EDIT: UPDATE: The maintainer of LuaSec confirms that this is called “SNI” (server name identification) and is available only in a later version of LuaSec than that which Vera is currently using. Now that I see it by that monicker, I am reminded that we are already aware of this–other developers have already run into this problem, and if memory serves, an upgrade to LuaSec on Vera is a much-requested firmware feature.

127

Just wanted to say that I was successful at using this plugin to access the Ambient Weather API where my personal weather station (PWS) loads data. (I have a WS-2902) Like many other people I was using the WeatherUnderground API/plugin, but when they killed the old API I decided I would look elsewhere and found this. You can request Ambient Weather’s API keys for free and it retrieves clean JSON using a URL of this type:

https://api.ambientweather.net/v1/devices/MACADDRESS?apiKey=***&applicationKey=***&limit=1

It returns results like this:
[{“dateutc”:1555194900000,“winddir”:266,“windspeedmph”:0,“windgustmph”:0,“maxdailygust”:6.9,“tempf”:48.4,“hourlyrainin”:0,“dailyrainin”:0.16,“weeklyrainin”:0.44,“monthlyrainin”:0.44,“totalrainin”:15.37,“baromrelin”:29.83,“baromabsin”:24.74,“humidity”:50,“tempinf”:68.2,“humidityin”:41,“uv”:1,“solarradiation”:168.7,“feelsLike”:48.4,“dewPoint”:30.63,“lastRain”:“2019-04-13T20:59:00.000Z”,“date”:“2019-04-13T22:35:00.000Z”}]

In case you try this, you need to use an expression of “response[1].tempf” to retrieve the outside temperature, for example - because the result is an array and the “limit=1” query parameter is asking for just the most recent data point.

Just posting this so someone else with the same problem might have an easier time finding this as an answer. Works great and thanks much!

3 Likes
128

OK, @lkas, the Github stable branch now contains a version of SiteSensor that can use curl as a conduit.

Here’s how you install:

  1. Go to the Github stable branch for the project: GitHub - toggledbits/SiteSensor at stable
  2. Click the green “Clone or download” button on that page, and choose “Download ZIP” from the pop-up. Save the ZIP file somewhere.
  3. Unzip the file.
  4. Upload each of the unzipped files using the uploader at Apps > Develop apps > Luup files. Make sure you turn off the “Restart Luup after upload” checkbox until the last file, then turn it back on. If you forget, no worries, just check the box and re-upload the last file.
  5. Wait for your system to reload.
  6. When it’s done, hard-refresh your browser.

Then, go into the subject SiteSensor instance’s control panel. Click the Advanced tab, and go to the Variables sub-tab. Find the “UseCurl” variable, and set it to “1”. The next time the sensor makes the query, it will use curl. If you have the “query only when armed” setting on, you can disarm and arm the sensor to stimulate this activity. Make sure you use the expression syntax in my previous reply.

129

It works great!
Thanks so much for your help, fast responses, and great software.
Larry K

1 Like
130

great versatile and handy app!
Based on SiteSensor I built a smart rainsensor as substitution for the good old app VirtualRainsensor (no longer working with the new Wunderground API).
I combined Sitesensor with a PHP script on my webserver, because I got problems to process the large and complex JSON response from Wunderground directly. This allows sophisticated procedures and it works fine!

1 Like
131

Is there a simple way to initiate the execution of the sitesensor per lua or per scene? Currently I use the way to disarm and afterwards arm the sitesensor.

132

Yes! I recently added a DoRequest action. It takes no parameters (just pass an empty table to the action):

luup.call_action( "urn:toggledbits-com:serviceId:SiteSensor1", "DoRequest", {}, devNumber )

1 Like
133

excellent!
Thank you!

134

Hello -
I’m having a great time with this plugin! As a few others are, I’m using it to call the new WU API to retrieve my weather station’s information.

However, I’m having a hard time calling the following item:
response.observations[1].imperial.obsTimeUtc

“Sensor” Response from the log. (It’s Null in the console)
10:05:38: SiteSensor: Eval #8: “response.observations[1].imperial.obsTimeUtc”=(“table”){ __type=“null” }

JSON RESPONSE:

{ “status”: { “valid”: 1, “timestamp”: 1557496538, “jsonStatus”: “OK”, “httpStatus”: 200 }, “response”: { “observations”: [ { “lon”: -83.48557281, “winddir”: 232, “country”: “US”, “humidity”: 68, “obsTimeLocal”: “2019-05-10 09:55:36”, “imperial”: { “windChill”: 51, “dewpt”: 41, “precipTotal”: 0.01, “precipRate”: 0, “pressure”: 29.88, “windGust”: 13, “windSpeed”: 4, “temp”: 51, “elev”: 702, “heatIndex”: 51 }, “neighborhood”: “Canton Center and Saltz”, “lat”: 42.31615448, “qcStatus”: 1, “epoch”: 1557496536, “softwareType”: “weewx-3.9.1”, “stationID”: “KMICANTO5”, “obsTimeUtc”: “2019-05-10T13:55:36Z” } ] } }

135

Huge extra points for posting the response data right off the bat! That’s a huge help!

The key you are after obsTimeUtc does not live within imperial, it lives at the same level, so the expression should be response.observations[1].obsTimeUtc.

If you put the response data through jsonlint.com, it becomes a bit more readable and makes that structure more obvious:

{
	"status": {
		"valid": 1,
		"timestamp": 1557496538,
		"jsonStatus": "OK",
		"httpStatus": 200
	},
	"response": {
		"observations": [{
			"lon": -83.48557281,
			"winddir": 232,
			"country": "US",
			"humidity": 68,
			"obsTimeLocal": "2019 - 05 - 10 09: 55: 36",
			"imperial": {
				"windChill": 51,
				"dewpt": 41,
				"precipTotal": 0.01,
				"precipRate": 0,
				"pressure": 29.88,
				"windGust": 13,
				"windSpeed": 4,
				"temp": 51,
				"elev": 702,
				"heatIndex": 51
			},
			"neighborhood": "Canton Center and Saltz",
			"lat": 42.31615448,
			"qcStatus": 1,
			"epoch": 1557496536,
			"softwareType": "weewx - 3.9 .1",
			"stationID": "KMICANTO5",
			"obsTimeUtc": "2019 - 05 - 10 T13: 55: 36 Z"
		}]
	}
}

Also, when posting code or JSON (or anything that should not be made “human readable” by these forums), put three backticks (``` – the character with the ~ key on most US keyboards) on a line by itself both before and after the code/JSON. That will keep the forum’s fancy formatting from breaking it.

1 Like
136

Thanks Patrick! I’m somewhat new to JSON so I appreciate the help! Makes sense!

Also, I appreciate the clues on quoting “code”. I was working from an iPad in an airplane and finally gave up! I’ll remember this for next time.

I’m considering a way to have Sitesensor query Weewx directly but this will work great for now!

1 Like
137

@wmichael3 . Thanks for sharing how you pulled the Ambient Weather!! I was wondering about what to do with my Ambient station now that WU has shut their API. I was thinking of hacking a new plugin, but this is FAR easier. I just started playing with SiteSensor has been a blast to play with and has solved several longstanding issues I have been wanting to resolve.

For the URL, I think the format should have a ‘?’ between the MACADDRESS and apiKey parameters. At least that is the only way I got it to work.

Also, based on the Ambient Device Data Specs. if you use just the /devices? endpoint vs. the /devices/MACADDRESS? endpoint, seems like the last data is automatically returned. Not sure if you have tried it yet.

138

@rigpapa , in the release notes for 1.10 you added the ability to increase the number of expressions > 8 by setting the Numexp state variable. I am not seeing this variable to edit in the UI. Is this something that needs to be set via Lua or can it be exposed via the Gui. Currently running 1.11

139

That’s odd. You can create it by going into the “New Service” tab under Advanced (copy-paste recommended here to ensure accuracy):

New Service: urn:toggledbits-com:serviceId:SiteSensor1

New Variable: NumExp

New Value: (enter whatever number > 8)

The click Add, wait a moment, and then click Reload Engine. While that’s happening, hard refresh your browser.

Be aware that the “Status” panel will only show the first 8 values. But the Settings tab should let you configure the full-number.

2 Likes
140

Thanks @rigpapa . That works within the plugin . It looks like the Imperihome ISS and Pleg still shows 8 sensor devices display or act upon. What is odd, is that in Pleg, triggers only show 8, however Device Properties, I have the extended values available.

141

No, That’s all normal Vera restriction, because the triggers have to be hard-coded in advance, so only 8–doesn’t change. The data is still there, as you’ve found, you just have to go at it differently. You could use the virtual sensor feature in SiteSensor, and trigger to that, for example.