SiteSensor will only do a string match on text responses, it won’t “deep inspect” or attempt to parse things out of it and make assignments of values to child sensors, as it does for the more structured JSON responses.
I have created a Site Sensor to pull down the JSON array of electricity prices for the next 24 hours from Octopus energy.
I want to parse the data to find when peak and cheap rates are, and so need to get more than 8 values out.
I have tried configuring the “response.result” data set to go into one of the Expression Values, but the code hangs “retrieving values”. Is there some other way to extract a set of values (or the entire response) that I have missed?
Thanks
Octo
Set the NumExp state variable on the SiteSensor to the number of expressions you need. But be aware only the first 8 will show on the status UI.
Many thanks for prompt response (and a great app), but I realise I would need 48+ expressions, and have to copy each expression and value out individually, so I’ll probably leave it for now 
Cheers
Requesting JSON…
I have an issue where my SiteSensor gets stuck in “Requesting JSON…” when my weatherstation breaks/crashes and stops sending data to Wunderground. The only way to free it back up is arm/disarm.
Any thoughts? I’ve tried messing with the timeout value with no change.
The old LuaSocket library has a number of bugs that can cause a hang regardless of the timeout. There are apparently also issues with LuaSec’s HTTPS negotiation. This is especially true on Edge (you didn’t mention what hardware you’re using, but just for info). You might try bypassing the ancient code by setting UseCurl to 1, which will have SiteSensor use curl rather the built-in socket/ssl support.
2 Likes
Interesting! I’m having similar strange issues around my TTS stuff (either Sonos or Alexa). Many times, a call out (to Azure, etc) causes a reload. Doesn’t matter if it is a scene, Reactor, etc. I have wondered if it was HTTP client related.
I will change this setting and see how it goes. Thanks for the quick response.
(I’m on a VeraSecure)
1 Like
Hello @rigpapa,
I’ve just started using octoprint for 3d printing and looking to query the if the printer is printing or has an error. I can query octoprint with sitesensor, but cannot find the correct value expressions. I looked here, but no combination of:
-printing
-flags.printing
-response.flags.printing
…are working.
Can you point me to a list of the correct value expressions? I couldn’t find this info on your github FAQ or in the recipes documentation…did I miss it?
Thanks in advance!
I had previously pasted the recipe from that link, (which appeared to work) but some of the sitesensor settings did not apply (such as the value expressions)…which I didn’t realize was unintentional.
I tried again and this time it seems to have worked properly fully populating.
Thanks!
Thanks @rigpapa for yet another awesome plugin! I was able to use SiteSensor with OpenWeatherMap to pull in current outdoor temp and humidity for use in a Reactor that will eventually control my whole-house humidifier. I’ve discovered that finding the ‘right’ humidity for a dwelling is a rather complicated matter, so there’s going to be a lot of math and sensor inputs!
I have some Awair Omni (air quality) sensors inside my home, and I want to use them to start the HVAC fan when temperatures or humidities of rooms vary widely, to even the house back out. For this the Omni provides an API in JSON format. I can hit http://192.168.0.55/air-data/latest in firefox, from my laptop, 192.168.0.69, and it returns this JSON string:
{“timestamp”:“2022-01-27T21:01:36.583Z”,“score”:83,“temp”:23.83,“humid”:51.60,“co2”:914,“voc”:1091,“pm25”:0,“lux”:0.0,“spl_a”:49.7}
I can curl it from the vera’s linux commandline as well and get the same string. But frustratingly it keeps getting http error 500 when sitesensor pulls it.
15:13:05: SiteSensor: Requesting JSON data
15:13:05: SiteSensor: “GET” “http://192.168.0.55/air-data/latest”, headers={ }
15:13:06: SiteSensor: Request returned no data, status 500
I don’t think I need to pass any headers, but firefox does show it is sending a few, like
| Accept | text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8 |
|---|---|
| Accept-Encoding | gzip, deflate |
| Accept-Language | en-US,en;q=0.5 |
| Connection | keep-alive |
| DNT | 1 |
| Host | 192.168.0.55 |
| Upgrade-Insecure-Requests | 1 |
| User-Agent | Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:96.0) Gecko/20100101 Firefox/96.0 |
So I set the headers in SiteSensor’s settings to:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
Connection: keep-alive
DNT: 1
Host: 192.168.0.55
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:96.0) Gecko/20100101 Firefox/96.0
And the next time it ran, it produced:
15:23:05: SiteSensor: Requesting JSON data
15:23:05: SiteSensor: “GET” “http://192.168.0.55/air-data/latest”, headers={ Connection=“keep-alive”, DNT=“1”, Accept=“text/html,application/xhtml xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8”, Host=“192.168.0.55”, [“Upgrade-Insecure-Requests”]=“1”, [“Accept-Language”]=“en-US,en;q=0.5”, [“Accept-Encoding”]=“gzip, deflate” }
15:23:35: SiteSensor: Request returned no data, status 500
So I think I’m just muddying things adding all those web browser headers. What should I try @rigpapa to get this GET getting more getfully?
A common reason for this is that the endpoint you are hitting actually redirects to another URL. Sometimes that is exposed in the browser and sometimes not.
On your SiteSensor, in Advanced > Variables, you will find a variable called FollowRedirects. Try setting that to 1 and retrying your request. It’s off by default, for security reasons, and because it’s rarely needed, but every once in a while…
I don’t think that’s is. When I hit it with curl, I saw no indication of any redirection:
$ curl http://192.168.0.55/air-data/latest -vvvvv
- Trying 192.168.0.55:80…
- Connected to 192.168.0.55 (192.168.0.55) port 80 (#0)
GET /air-data/latest HTTP/1.1
Host: 192.168.0.55
User-Agent: curl/7.79.1
Accept: /
- Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/json
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< Connection: Keep-Alive
< Transfer-Encoding: chunked
< - Connection #0 to host 192.168.0.55 left intact
{“timestamp”:“2022-01-27T23:58:42.997Z”,“score”:86,“temp”:22.87,“humid”:46.50,“co2”:799,“voc”:1868,“pm25”:0,“lux”:0.0,“spl_a”:47.8}
But it won’t hurt to try enabling redirection as you described, so I did that and took the headers off and it still got a 500:
18:09:21: SiteSensor: Requesting JSON data
18:09:21: SiteSensor: “GET” “http://192.168.0.55/air-data/latest”, headers={ }
18:09:22: SiteSensor: Request returned no data, status 500
While I was looking in advanced > variables, I noticed there was one to use curl, as opposed to LuaSocket I assume) and I enabled that. Now the log window shows
18:13:49: SiteSensor: Requesting JSON data
18:13:49: SiteSensor: curl -m 10 -o ‘/tmp/Q_SiteSensor_786683.txt’ -L ‘http://192.168.0.55/air-data/latest’
18:13:50: SiteSensor: Response is 131 bytes in “/tmp/Q_SiteSensor_786683.txt”
And while that txt file does not exist, there’s a .lzo of it that does contain the json I was expecting.
so it looks like my luasocket is broken? I bet that has a lot of other fallout!
UPDATE: I am now able to pull values from the response. object! horray! I’m kind of interested in fixing luasocket. Especially because it works without forcing curl to fetch data from openweathermap. Odd that it wouldn’t work locally. But I’m not sure it’s a problem with SiteSensor at this point. just something flakey with vera’s lua stack
Hi,
I am currently attempting to test the SiteSensor plugin on my VeraEdge 1.7.5185 (7.31). I got a failure and don’t see the reasons of it.
I have enabled the “UseCurl” option, as the other method was not working at all. I performed a simple test with the API request https://api.ipify.org/?format=json, which worked well on my VeraEdge only in “UseCurl” mode.
Next, I attempted to use https://api.open-meteo.com/v1/forecast?latitude=60.63&longitude=28.57¤t_weather=true
“Sensor is triggered when: URL unreachable or server replies with error”
and received the following log:
22:51:05: SiteSensor: Requesting JSON data
22:51:05: SiteSensor: curl -m 60 -o ‘/tmp/Q_SiteSensor_4.txt’ -L -0 ‘https://api.open-meteo.com/v1/forecast?latitude=60.63&longitude=28.57¤t_weather=true’
22:51:05: SiteSensor: Request failed, nil
22:51:05: SiteSensor: Request returned no data, status 500
I have tried various options in Advanced → Variables, but have been unsuccessful in resolving the issue.
The same test works well with curl on my PC running Linux, and I am able to retrieve the correct JSON file without errors:
{“latitude”:60.62441,“longitude”:28.574387,“generationtime_ms”:0.3389120101928711,“utc_offset_seconds”:0,“timezone”:“GMT”,“timezone_abbreviation”:“GMT”,“elevation”:8.0,“current_weather”:{“temperature”:-4.9,“windspeed”:19.1,“winddirection”:275.0,“weathercode”:1,“time”:“2023-03-12T19:00”}}
I have even reset my Vera to default settings and reinstalled the plugin, but without success. Is this problem specific to the weather site or is there an issue with my VeraEdge?
Thank you in advance,
Denis
Although the Edge is the same generation as the Plus, its firmware contains libraries that are not as up-to-date (all things being relative) as those on the Plus, and there are many URLs the Edge cannot reach because of this (while a Plus may still be able to).
Where you need to test your curl command on the command line is on the Edge itself. If you can successfully get a response from curl when you run it from a shell on the Edge (i.e. when you ssh into your Edge and run the curl command there), then you have a problem with your SiteSensor config. If the test fails, however, that indicates that your Edge is unable to negotiate the SSL connection with the remote server, and that will be about all for that discussion (your Edge is another step on its way to its inevitable role as a door stop).
EDIT: When I tried your URL on my Plus, it gives me a certificate warning that the site has an invalid or expired certificate. I checked the cert, and is seems OK, so I suspect it’s merely the SSL library complaining that a CA certificate on the chain of their server certificate (a free LetsEncrypt cert) is expired. That, again, is because of the ancient, rarely-update firmware libraries and packages on these systems. Try setting CurlOptions to -k to work around it. It works on my Plus.
It works! Thank you!
1 Like