ll.Table2Json and ll.Json2Table
complete
WolfGang Senizen
Functions to convert from lua table to json string would be amazing.
ESPECIALLY if they can handle vector uuid and quaternion casting correctly.
ll.Table2Json
Recursively convert a table to json.
Some values in a table may not be convertible like functions, and should probably be skipped.
local data = {1,2,{a="b"},{c={d="e"}}}
local json = ll.Table2Json(data)
ll.OwnerSay(json)
Would output something similar to
[1,2,{"a":"b"},{"c":{"d":"e"}}]
ll.Json2Table
Convert a json string to a table.
local json = '[1,2,{"a":"b"},{"c":{"d":"e"}}]'
local data = ll.Json2Table(json)
Would create a table similar to
{1,2,{a="b"},{c={d="e"}}}
Casting of vector quaternions uuids booleans etc
It would solve ALLOT of problems if these methods handled casting of certain datatypes for users.
Avoid lsl's
(vector)llList2String(list,0)
problemP.S.
I have made a "crappy" lua function that mocks ll.Table2Json mostly for debugging purposes
Log In
H
Harold Linden
complete
According to Rider this was rolled out today,
lljson.encode()
and lljson.decode()
appear to be working. Please file a Canny if you find any issues with it!H
Harold Linden
in progress
H
Harold Linden
Just a quick update, we've got a JSON serialization module based on
cjson
called lljson
working internally, and we wanted feedback on how things will work:* Currently, an empty table is treated as an empty JSON object due to
cjson
's existing behavior, but you can say you want a table to be serialized as an array by giving it an lljson.array_mt
metatable, or by passing lljson.empty_array
.* All built-in data types other than functions are supported for serialization, but we don't do automatic casting of strings that look like vectors or UUIDs on deserialization for the reasons Signal mentioned below
* There's no pretty-printing so people get a memory-efficient JSON blob by default, but we might be able to hack pretty-printing into
cjson
if enough people need it* Values that can never be serialized (like functions) cause an error to be thrown. This is consistent with the JSON libraries in most other languages.
* Only
string
keys are allowed, since that's all JSON allows. An error is thrown otherwise. Most JSON libraries do not do automatic coercion and cjson
(which lljson
is based on) also does not.I've attached a big chunk of our tests for
lljson
to give you an idea of how it currently behaves in our dev version.
WolfGang Senizen
Harold Linden
Awesome!!
The only bit that's a downside is the error on function, means serializing an object is going to be awkward, unless you guys were maybe willing to add a
__tojson
style metamethod as some other languages do, or a flag to skip them functions.Though that can probably be worked around with strict metatable usage I guess.
maybe testing something like this
local Funcs = {}
Funcs.__index = Funcs
function Funcs.test()
ll.OwnerSay("Test!")
end
local obj = setmetatable({a=1},Funcs)
obj.test()
assert(lljson.encode(obj) == '{"a":1}')
But that can always be discussed later, Getting this stuff at least internally addressed so quick is cool. Thanks.
H
Harold Linden
WolfGang Senizen
> The only bit that's a downside is the error on function, means serializing an object is going to be awkward [...] though that can probably be worked around with strict metatable usage I guess.
Yeah, anything on the metatable (which is hopefully where most functions on a table live) does not get picked up by
lua_next()
iteration, so you should be good in the general case. If you have a custom __iter
or __index
it's not going to be called during encoding.> unless you guys were maybe willing to add a
__tojson
style metamethod as some other languages doGood point, that seems reasonable and we'll look at doing that. Do note that you'd want to avoid anything non-trivial inside
__tojson()
, as anything that coroutine.yield()
s or runs long enough it gets stopped by our scheduler would potentially be a problem. We can't gracefully pause your code in cases like that, because the VM won't know how to pick back up where it left off if your function was called within a C++ function we wrote.> or a flag to skip them functions
I think we'll have to revisit that later. Technically
cjson
has an option for it, but we removed the config methods because the cjson
configuration is usually VM-global, and we didn't want to give every script a new copy of the underlying read-only globals table.H
Harold Linden
Another question is what to do about
buffer
s. I'm inclined to treat those as unserializable because JSON doesn't have a native way to serialize binary data (JSON strings are unicode,) but I could also see converting them to Base64 or something by default.WolfGang Senizen
Harold Linden
Thanks for the info
Did you see my extra bit about testing on tables with a set metatable I edited it into my comment after so you may not have had it when you replied.
H
Harold Linden
WolfGang Senizen:
Yep, I saw that,
__index
on metatables and whatnot are totally ignored during serialization, but for clarity:-- __index and friends are ignored on metatables
local SomeMT = {}
function SomeMT.whatever(...)
error("Placeholder function called")
end
SomeMT.__index = SomeMT
assert(lljson.encode(setmetatable({foo="bar"}, SomeMT)) == '{"foo":"bar"}')
WolfGang Senizen
Harold Linden
Great!
H
Harold Linden
Cool, we'll push this out sometime next week then!
Signal Linden
Re: "Casting of vector quaternions uuids etc", we're not going to do much fancy auto-casting on deserialization since that's caused us a lot of trouble previously. JSON is missing support for a lot of valid values in LSL (infinity, NaN, UUIDs, vectors, quaternions, etc.)
It would be surprising if a string field within JSON that happened to contain "<1,1,1>" got magically transformed into a vector even though it was meant to be treated as a string. Without some indication as to what a script wants for a field, we don't want to magically mess with the data in ways that may be unwanted. As an existing example of that,
llJsonGetValue()
will silently trim surrounding spaces for string values before giving you them, even though the spaces might be important to you. We won't do things like that in the new functions.Note that we will serialize custom datatypes (i.e.
uuid
becomes a string
,) but converting from string back to a custom datatype is dangerous territory for a general-purpose function. Luckily SLua is flexible enough that you can trivially write a deserialization wrapper to walk the deserialized data and convert it according to a schema if it's important to you.We're also looking at exposing other serialization schemes that would allow 100% round-trippable data serialization from SLua scripts without any explicit casts on deserialization, but that might take some time. The scheme we currently use within the VM for serialization script states would work, but it assumes the input comes from a trusted source.
Signal Linden
planned
Absolutely! We're planning to do something like this based on OpenResty's fork of lua-cjson.
Kadah Coba
A couple pure lua JSON libs I found
A c implementation
SamanthaTekcat Resident
It would be also helpful to have Functions for Tables similar to the llJsonSetValue and llJsonGetValue functions in Lua
WolfGang Senizen
Working with SamanthaTekcat Resident yesterday
We came up with
setTableVal
local function setTableVal(tab,path,val)
for k,v in pairs(path) do
if k == #path then break end
if type(tab[v]) == nil then tab[v] = {} end
tab = tab[v]
if type(tab) ~= "table" then return false end
end
tab[path[#path]] = val
end
And
getTableVal
local function getTableVal(tab,path)
for k,v in pairs(path) do
tab = tab[v]
if tab == nil then return nil end
end
return tab
end
As helper lua functions for those
They aren't identical as they modify the table in place, and don't quite support the
JSON_DELETE
type of functionality, you can pass nil
but that's a bit different