DataYours - filtering out wrong values

Plugins & Plugin Development DataYours
#21

Hi akbooer,

when the json file is posted to “whisper-editor.lua” to update some records, the update will change the record value using the “datetime” key ? If the key (datetime) doesn’t exist will it be added ?

tnks

donato

#22

A null timestamp in the [timestamp, datavalue] pair of the datapoint structure means that pair will be ignored in the data update. A valid timestamp with a non-null datavalue will either overwrite or create an entry with that value, provided that it refers to a time before the current time. I believe the database is using local time stamps rather than UTC. The timestamp will be rounded according to the schema for the file archives.

#23

A null timestamp in the [timestamp, datavalue] pair of the datapoint structure means that pair will be ignored in the data update. A valid timestamp with a non-null datavalue will either overwrite or create an entry with that value, provided that it refers to a time before the current time. I believe the database is using local time stamps rather than UTC. The timestamp will be rounded according to the schema for the file archives.[/quote]

ok tnks akbooer,

this is useful for me in the case I need to sync two wsp files relative to the same watched variable and in general to know how the update is working .

tnks again

#24

Hello,

I want to ignore values outside the boundary.

Should this work?

if metric: match "Temperature" then local v = tonumber(value) if v < -50 then value = nil elseif v > 150 then value = nil end end

Asking here as it’s not so easy to generate incorrect data :stuck_out_tongue:

#25

[quote=“Sjoerd, post:24, topic:191815”]I want to ignore values outside the boundary.

Should this work?[/quote]

You’re talking about the code within the relay() function of the run() function within the L_DataUser.lua file?

Rather than changing the metric value, you should return a nil metric name (this conforms to the Lua pattern for an iterator function, which is what it is.)

  local function relay ()

    if metric: match "Temperature" then
      local v = tonumber(value)
      if (v < -50) or (v > 150) then 
        metric = nil
      end
    end

    local m = metric
    metric = nil
    return m, value, time   -- first call returns metric, second one nil
  end
Asking here as it's not so easy to generate incorrect data :P

Really? Surely it’s as simple as using the GUI to change the required device variable to any value you like. The routine watching the variable will pick up the changes and pass it along through the processing chain.

#26

Having re-read my response, and taking a closer look at my original code which showed this as an example (!), it seems to me that it would be more straight-forward to write it like this, rather than mess with the iterator function itself…


function run (metric, value, time)  
  
  local function relay ()
    local m = metric
    metric = nil
    return m, value, time   -- first call returns metric, second one nil
  end
  
  -----
  --
  -- user-defined processing goes here
  -- change metric name, value, or time as required, or
  -- set metric to nil if you don't want to pass it on
  --
    
  if metric: match "Temperature" then 
    local v = tonumber(value)
    if (v < -50) or (v > 150) then 
      metric = nil
    end
  end
 
  -----
  
  return relay
end
#27

Thanks,
Implemented it in this way

[quote author=akbooer link=topic=37119.msg333093#msg333093 date=1509448481]

[quote=“Sjoerd, post:24, topic:191815”]

Asking here as it’s not so easy to generate incorrect data :stuck_out_tongue:

Really? Surely it’s as simple as using the GUI to change the required device variable to any value you like. The routine watching the variable will pick up the changes and pass it along through the processing chain.[/quote]

Indeed very simple, could have think about it myself…

#28

After some playing around I Wanted to log which data is droped.

Code I used:

[code]function run (metric, value, time)

local function relay ()
local m = metric
metric = nil
return m, value, time – first call returns metric, second one nil
end
if metric: match “Temperature” then
local v = tonumber(value)
if (v < -50) or (v > 150) then
luup.log("DataUser : dropped " … metric … “=” … value)
metric = nil
end
end
if metric: match “Humidity” then
local v = tonumber(value)
if (v < 1) or (v > 99) then
luup.log("DataUser : dropped " … metric … “=” … value)
metric = nil
end
end
return relay
end
[/code]

It is working I do not have incorrect data in the database,
However I found in the log:

50 11/04/17 23:25:25.430 luup_log:9: DataUser : dropped Vera-50104436.105.urn:upnp-org:serviceId:TemperatureSensor1.CurrentTemperature=188 <0x72271520> 01 11/04/17 23:25:25.431 eLuaInterface::CallFunction_Variable func: DataWatcherCallback Device_Variable 105 urn:upnp-org:serviceId:TemperatureSensor1:CurrentTemperature failed [string "module ("L_DataUser", package.seeall)..."]:33: attempt to index local 'metric' (a nil value)e <0x72271520>

#29

Yes, it’s your coding error: if it’s an invalid temperature, then metric is set to nil, but then you go on and test to see if it matches humidity…

…at the very least, your if … end if …end, should be if … elseif … end

Although there are other, possibly better, solutions too.

#30