How to - LUA Script to extract particular data / value from string variable (JSON API) HTTP Requests

I am using OpenWeather in this example however this LUA Script should work for any JSON data you have managed to download via a HTTP Request, from an online API server etc.

Currently in Ezlogic you cannot use a relatively simple one line “Expression Code” to extract a particular piece of data from a variable.

For example say you have a Meshbot rule to send a HTTP Request to the OpenWeather service and you store that downloaded weather data into a local string variable. It may look something like this:

And say from all that data, we want to extract only the data / value for Cloud Coverage.

"clouds":{"all":3}

We then want to store that particular data / value into another local integer variable, so we can then use that value as a trigger for our Meshbot rule(s) etc.

Note - This script example is for use when the source variable is a “string” and the target variable is an “integer” e.g. for extracting numbers.

So how can we do this ?

First you need to get a working HTTP Request URL for the OpenWeather service (or any other online JSON data API you like).

These API calls can be used for free, I am using the “Current Weather” one in this example.

Current Weather

https://api.openweathermap.org/data/2.5/weather?lat={lat}&lon={lon}&appid={API key}

5 day weather forecast

https://api.openweathermap.org/data/2.5/forecast?lat={lat}&lon={lon}&appid={API key}

You just need to add your own latitude and longitude to the HTTP command and also your OpenWeather API key. If you don’t have an OpenWeather API key already? This page here might help you get started.

You can then test your HTTP command in a web browser first and it should then return some weather data.

So we first need to create our local variables where the weather data and also cloud coverage data will be stored and held. To do this in Ezlogic web GUI, go to Automation → Advanced Scripting → Variables area.

Create a new “string” variable and call it “OpenWeather” and then create a new “integer” variable called “OpenWeatherCloudCover”. Note if you cannot Save, put a 1 in the “Variable Value” field for now.

OK so now we have two variables to store our data, we can look at creating our LUA script to extract the particular bit of data we are interested in, which is the cloud coverage value.

In Ezlogic go to Automation → Advanced Scripting → LUA Scripts. Create a new script give it a name and paste in the LUA script below.

NOTE - Thanks to Alexey an Ezlo developer for providing this LUA script to us.
You can also find this LUA Sctipt on gitHub here.

-- Oleksii.Kirdin, 2023-06-12, fixed 2023-06-22
-- Anonymous Plugin (aka Lua script) for Ezlogic.

-- Get a stringified json from one variable, parse it to a Lua table, extract some value by path, save it to another variable with value type respect.
-- Meshbots/Scenes/Global Variables/HTTP Request Parsing.
-- The script can be used as an action in a meshbot after the sendHttpRequest/cloudAPI calls with SaveResult attribute
-- to post-process JSON value: extract a value to another expressions variable with type information.
-- After that, the variable with type can be used as a trigger in meshbots.

-- OpenWeather - a variable of type "string";
-- OpenWeatherCloudCover - a variable of type "float" or "int";
-- path_to_extract - path inside of the JSON stored in OpenWeather variable to fetch data by.
-- Result: saves extracted numeric value to the target variable.

local logging = require "logging"
local json = assert(require "json", "Could not load json library")
local scenes = require "scenes"

local variables = {
    source = "OpenWeather",
    target = "OpenWeatherCloudCover"
}

local path_to_extract = {"clouds", "all"}

local function extract_by_path(data, path)
    local value = data
    for _, key in ipairs(path) do
        logging.debug("key: " .. tostring(key))
        value = value[key]
        logging.trace(value)
    end
    return value
end

local function extract_and_save(variable_from, variable_to, path)
    logging.debug("HTTP Response variable: " .. variables.source)

    local http_response_variable = scenes.get_expression_value(variable_from)

    if http_response_variable.value ~= nil then
        logging.debug("Source variable has value: " .. tostring(http_response_variable.value))

        local data = http_response_variable.value

        local res = json.decode(data)
        logging.debug("After json.decode")
        logging.trace(res)

        local found_value = tonumber(extract_by_path(res, path))

        local success, saved_value = pcall(scenes.get_expression_value, variable_to, {compact = false})
        logging.trace("pcall.success: " .. tostring(success))
        logging.trace("pcall.saved_value: " .. json.encode(saved_value))

        local target_value = {
            value = found_value,
            value_type = saved_value.value_type or "float",
            metadata = saved_value.metadata or {}
        }

        logging.debug("Save to target variable: " .. variable_to)
        scenes.set_variable(variable_to, target_value)
        return true
    else
        logging.error("Source variable has no value")
        if http_response_variable.error ~= nil then
            logging.error("Variable.error: " .. tostring(http_response_variable.error))
        end
        return false
    end
end

return extract_and_save(variables.source, variables.target, path_to_extract)

And finally we need to now create our Meshbot rule. The rule will run on a time interval say every hour, it will run the HTTP command to get all the weather data and then it will run the LUA script to extract only the cloud coverage value.

So go to Automations → Meshbots and click the “Create new Meshbot” button and select a “Local” Mesbot type.

Give your Meshbot a name I called mine “GET - Open Weather Cloud Data - Local Variable”

TRIGGER - The trigger is just an Interval of every hour, you can set your Interval as you wish.

ACTION 1 - The first action is to send the HTTP Request to the OpenWeather online API to get all the weather data and we will store it into the string variable called “OpenWeather”.

Select the Node “HTTP Request” and “GET” is fine for this request. Paste in your OpenWeather HTTP command.

Ensure you select “Execute next action sequentially if this action succeeds” from the “Execution Policies” menu on the right hand side.

Then at the bottom select “Save output to local variable” and select your “string” variable named “OpenWeather”.

ACTION 2 - In action 2 we run our existing LUA script.

Select the “LUA Script” node then “Select Existing Script” then select the script you created earlier.

Again ensure you select “Execute next action sequentially if this action succeeds” from the “Execution Policies” menu on the right hand side.

And we probably want to add a small delay to give Action 1 sometime to complete, I used 5 seconds.

Now save your Mesbot rule and we can press the “Play” button next to it so we can test it is working.

Now we can go back to the Variables area of the web GUI and see if we have the data now stored ? You may need to refresh the browser page or click in and out of that area for the Cloud Cover value to be seen.

And finally you can then create another new Meshbot rule and use the “Cloud Cover” local integer variable as your trigger.

Select the “Local Variable” node and then select the “OpenWeatherCloudCover” variable, you can then use any logic operators you wish, in this example I just used Greater than 70.

EDITING THE LUA SCRIPT -

The only part of the LUA script you need to edit to suit your own needs are the local variables and the path of the data you want to extract.

“source” is the string variable where all the weather data was stored.
“target” was our integer variable where we stored just the cloud cover value.

And the path for that particular bit of data / value was - {"clouds", "all"}

local variables = {
    source = "OpenWeather",
    target = "OpenWeatherCloudCover"
}

local path_to_extract = {"clouds", "all"}

So for example say if you wanted to extract the current “humidity” value instead your LUA script would look like this instead, note I am using a different integer variable called “OpenWeatherHumidity” and the path looks different also.

local variables = {
    source = "OpenWeather",
    target = "OpenWeatherHumidity"
}

local path_to_extract = {"main","humidity"}

To work out the JSON path to the value you wish to extract you need to Pretty Print the JSON output from the Online API service. There are various websites for doing that or I used Visual Studio Code application and a Prettify JSON Extension.

So I could then clearly see what the path to the humidity value was etc.

image

FINAL THOUGHTS -

I have also tested this LUA script with another online JSON API service, one to do with Covid Stats and I was able to extract a particular piece of data / value to another integer variable. So this LUA script should work for extracting data values from any online JSON API service and that stored data in your string variable.

Hopefully in the future Ezlo will make this easier and give us the ability to use a more simple one line “expression code” to extract the desired value from a local variable and then this LUA script would no longer be required. But until then this is the only way to do it currently.

4 Likes

Example script # 2.

In this example our source variable is of “string” type and our target variable is also of “string” type.

I am going to use a WAN IP as an example, say you have a static VPN WAN IP that you pay for, we can monitor if that IP address changes, thus meaning your VPN tunnel has disconnected and you are now showing your ISP assigned WAN IP address etc.

1 .Create two new Variables to store the string data. On the menu go to “Automation” → “Advanced Scripting” → “Variables”

Name the first string Variable “IPAPI”

image

Name the second string Variable “myIP”

image

  1. Create the LUA script which will extract only the WAN IP address from the source variable / JSON data. On the menu go to “Automation” → “Advanced Scripting” → “LUA Scripts”

Create a new script call it “Extract WAN IP” and paste in the following code.

Note - This LUA script is also available on Github here.

-- Oleksii.Kirdin, 2023-06-12
-- Anonymous Plugin (aka Lua script) for Ezlogic.
-- Result: saves extracted string value to the target variable.

local logging = require "logging"
local json = assert(require "json", "Could not load json library")
local scenes = require "scenes"

local variables = {
    source = "IPAPI",
    target = "myIP"
}

local path_to_extract = {"ip"}

local function extract_by_path(data, path)
    local value = data
    for _, key in ipairs(path) do
        logging.debug("key: " .. tostring(key))
        value = value[key]
        logging.trace(value)
    end
    return value
end

local function extract_and_save(variable_from, variable_to, path)
    logging.debug("HTTP Response variable: " .. variables.source)

    local http_response_variable = scenes.get_expression_value(variable_from)

    if http_response_variable.value ~= nil then
        logging.debug("Source variable has value: " .. tostring(http_response_variable.value))

        local data = http_response_variable.value

        local res = json.decode(data)
        logging.debug("After json.decode")
        logging.trace(res)

        local found_str = extract_by_path(res, path)
        logging.debug(found_str)

        -- fetch ONLY in full mode, not in compact:
        local success, saved_value = pcall(scenes.get_expression_value, variable_to, {compact = false})
        logging.trace("pcall.success: " .. tostring(success))
        logging.trace("pcall.saved_value: " .. json.encode(saved_value))

        local target_value = {
            value = found_str,
            value_type = saved_value.value_type or "string",
            metadata = saved_value.metadata or {}
        }

        logging.debug("target_value: " .. json.encode(target_value))
        logging.debug("Save to target variable: " .. variable_to)
        scenes.set_variable(variable_to, target_value)
        return true
    else
        logging.error("Source variable has no value")
        if http_response_variable.error ~= nil then
            logging.error("Variable.error: " .. tostring(http_response_variable.error))
        end
        return false
    end
end

return extract_and_save(variables.source, variables.target, path_to_extract)
  1. Create a Meshbot rule to obtain your current WAN IP address on a time interval. I named mine “GET My WAN IP Address”.

Trigger:

Action:

There are various websites online that you can use to send a HTTP request to them to then receive your current WAN IP address in JSON format. I am using https://ipv4.jsonip.com/ currently.

Action #1

Use HTTP Request and GET and use this URL https://ipv4.jsonip.com/

Tick the “Save output to local variable” option and select the “IPAPI” variable.

Also ensure you select “Execute next action sequentially if this action succeeds” from the “Execution Policies” menu on the right hand side.

Action #2

Select the “LUA Script” node then “Select Existing Script” then select the script you created earlier.

image

Again ensure you select “Execute next action sequentially if this action succeeds” from the “Execution Policies” menu on the right hand side.

And Save the Meshbot rule.

We can now test the rules Actions by pressing the Play button next to it and this should then populate our string variables with data.

Go back to the Variables area and check your string variables now contain data.

Source Variable “IPAPI” -

Here we can see the API JSON data from the HTTP Request URL.

image

Target Variable “myIP” -

And in the target variable you should see that the LUA script has just extracted the “ip” portion only

image

  1. Create another new Meshbot that checks if your WAN IP address has changed and carry out some Actions like notifications or anything else you like.

Trigger:

In the “Node Type” select “Local Variable” then select “myIP” in " Comparator" select “Not equal (!=)” and then in the “Value” enter your static VPN WAN IP address etc.

I also used the Function option “For At Least” and set it to 10 minutes. This means that this trigger must be true and sustained for at least 10 minutes before my rules Actions will be carried out.

Actions:

The actions can be anything you like, send yourself a notification, make some lights flash, send a TTS announcement etc. Or even run a command to start the VPN client again on your router.

To test this second Meshbot is working, you could disconnect your VPN tunnel and thus your WAN IP address will go back to your ISP assigned WAN IP address etc and this should then trigger your Meshbot rule.

3 Likes