Best way to parse JSON or XML

Plugins & Plugin Development General Plugin Discussion
1

I have some ideas for Vera Plugins, but before I get started I was wondering if there’s a Luup/LUA library available for parsing XML or JSON?

Would be a lot nicer then using the string library as most plugins seems to do now. And I don’t think that writing a JSON or (worse) a XML parser would be a nice project to start learning LUA with ;).

2

I really searched for parsing JSON via the search and could not find any results. But searching via Google got me to this thread:

http://forum.micasaverde.com/index.php?topic=2423.0

But there’s still no answer on how Vera parses JSON internally. It’s used so widely in the Vera system that you would suspect that LUA has some kind of JSON parser built in.

3

My /usr/lib/lua directory has the lxp library, which is apparently an
XML parser. Good chance that it is on everyone else’s Vera too.

I don’t see a JSON parser library there. They exist (here is one: Jeffrey Friedl's Blog » Simple JSON Encode/Decode in Pure Lua) but you would have to install it yourself, which makes distributing plugins based on it a pain.

I haven’t used any of these libraries personally.

The MiOS engine, LuaUPnP, is a compiled C++ executable. It presumably has its own JSON and XML libraries, which it doesn’t expose to the Lua interpreter that is also built into the executable.

4

The lxp library is on all Veras.

Not really, at least on UI5. The library (if it’s a Lua file,) can be added to the plugin as any other Lua file.

5

It’s still a pain for as long as apps.mios.com doesn’t let two different plugins include a file with the same name. Renaming it to be different for each plugin doesn’t scale and wastes storage.

Apps.mios.com is evolving towards a package management system. Package management is hard to do properly. before apps.mios.com evolves further, give some serious thought to using the one that comes with OpenWrt.

6

It’s still a pain for as long as apps.mios.com doesn’t let two different plugins include a file with the same name. Renaming it to be different for each plugin doesn’t scale and wastes storage.

Apps.mios.com is evolving towards a package management system. Package management is hard to do properly. before apps.mios.com evolves further, give some serious thought to using the one that comes with OpenWrt.[/quote]

Totally agree… Vera needs to leverage more open source community projects, AND allow use to use them in a supported way.

Anyway, back on topic :slight_smile:

@Meinaart

I hope you can get a JSON parser working. XBMC uses JSON notifications and I’d love to have the Vera integrate with XBMC.

Chixxi said he was working on an XBMC plugin, which would require a JSON parser, so I suggest you ping him to see if you can collaborate.

@ Everyone

Don’t be afraid to contact MCV directly. I suspect the more coders we have reaching out to MCV directly (phone, email ,etc) to get help building something that can benefit the entire community, the more they might be willing to help. The key here is to explain HOW what you are doing can benefit the entire community, and make their lives easier in the long run.

Example: “I’m building a JSON receiver/parser for the Vera and would like a small bit of assistance. I promise not to take up much of your time if youd be kind enough to assist. This plugin will benefit the entire community as there are many apps that communicate using JSON and the list is growing daily as it has become the defacto communication standard for apps. Having this available to the community will allow Vera to integrate with apps like: ”

7

I write a plugin that use an external json parser module (come from here json/json.lua at master · luaforge/json · GitHub ).
Maybe my plugin could check (in /usr/lib/lua) if this module exists and download it during the first start ?!
This way each plugin that use this module will find it …
Is there a better solution ?

8

I will add Lua CJSON on Vera. I analyzed several Json encoders/decoders and this is the fastest.

Json modules comparison:
http://lua-users.org/wiki/JsonModules

9

Just checking if Lua CJSON has been added to the vera3? I am trying to run some test code

-- Module instantiation
local cjson = require "cjson"
local cjson2 = cjson.new()
local cjson_safe = require "cjson.safe"

-- Translate Lua value to/from JSON
text = cjson.encode(value)
value = cjson.decode(text)

and I am getting an error that the filed is nil so my guess is that it has not been added, but it could be user error, so I thought I would check before giving up.

10

Sorry, I didn’t get to add it yet. I recommend using other library until then.

11

okay, thanks. Thats what I ended up doing.

12

now is 2015 … and there is still no json parser on the vera at all ?

13

Yes there is. They went for dkjson. All you need is:

local json = require "dkjson"
14

for my plugins, I got around the lack of json parser by using the following function…

function decode_json(json)
	if (not json) then 
		return nil
	end
	local str = {} 
	local escapes = { r='\r', n='\n', b='\b', f='\f', t='\t', Q='"', ['\\'] = '\\', ['/']='/' } 
	json = json:gsub('([^\\])\\"', '%1\\Q'):gsub('"(.-)"', function(s) 
		str[#str+1] = s:gsub("\\(.)", function(c) return escapes[c] end) 
		return "$"..#str 
	end):gsub("%s", ""):gsub("%[","{"):gsub("%]","}"):gsub("null", "nil") 
	json = json:gsub("(%$%d+):", "[%1]="):gsub("%$(%d+)", function(s) 
		return ("%q"):format(str[tonumber(s)])
	end)
	return assert(loadstring("return "..json))()
end

it is called like:

local data = decode_json(json_string)

The returned value in the “data” variable is a lua table containing the key/value elements in the json string. Be aware that any elements in the json string that evaluate to the json null value are not translated into the data table, so value checking is required.

If programming for UI7, all versions have included the dkjson library…

local JSON = require("dkjson")
local toencode = JSON.encode(lua_table)
local todecode =JSON.decode(json_string)
15

okay dkjson seems to work …
but what do i do now with UI5 users ?

load both would be not very professional

16

any idea whats wrong with that:

	if (luup.version_major < 7) then
	    local endecoder = require("akb-json");
	else
	    local endecoder = require("dkjson");
	end

that somehow gives me a LuImplementation::StartLua skipping device 245 implementation valid 1 <0x2b6a5680>

17

[quote=“nullx8, post:16, topic:171469”]any idea whats wrong with that:

	if (luup.version_major < 7) then
	    local endecoder = require("akb-json");
	else
	    local endecoder = require("dkjson");
	end

that somehow gives me a LuImplementation::StartLua skipping device 245 implementation valid 1 <0x2b6a5680> [/quote]

You are defining the endecoder variable inside the if/then/end… it is scoped for use only inside that statement… after the if/then/else/end it goes out of scope and causes a crash when you try to use it. use:

	local endecoder = nil
	if (luup.version_major < 7) then
	    endecoder = require("akb-json");
	else
	    endecoder = require("dkjson");
	end
18

still same error

LuImplementation::StartLua skipping device 245 implementation valid 1

if i remove the if/else/then
and just do
local endecoder = require(“akb-json”)

it works like a charme … it also works in test lua

just the startup seem to have a problem with this statement

looks like “luup.version_major” is not avaiable in startupLua

19

try:

local endecoder = assert(require("dkjson"))
if (type(endecoder) ~= "table") then
    endecoder = require("akb-json")
end
20

This topic should be pinned!
I spent hours looking for how to parse a json string… until I found this magical line

local json = require("dkjson")