Transparent, native, Lua libary loading and UI4/UI5 incompatabilities

Regarding bug #2000:

On UI4, loading Lua files was a hack because it used a modified uploading script which didn’t compress the Lua files. This hack has been added in version 1.1.1169 IIRC. Installing a plugin from the Marketplace compressed the Lua files (see L_WapUI.lua and L_ThermostatMinMax.lua).

On UI5 the uploading script has been reverted to the pre 1.1.1169 version. I like that the Lua files are now consistent with the rest of the files regarding compressing and would like to remain this way.

Tomorrow I will write a function to load a file regardless of whether it’s compressed or not, which you can add in the startup function of the plugins that use Lua files.

In standard Lua programming, there are three elements that can be manipulated wrt Library loading. These roughly translate into

a) the Path to look at for the content, which is an array of strings (often pre-populated by LUA_PATH as an environment)
b) a set of one-or-more Loaders which are called, in order, to attempt to resolve the named Library being requested by some means
c) a Cache of previously resolved [in memory] Libraries for this Lua Context.

These are kept in [tt]package.cpath[/tt], [tt]package.loaders[/tt] and [tt]package.loaded[/tt] (resp).

Early versions of UI4 couldn’t use .lua libraries as it hadn’t instantiated a correct/default [tt]LUA_PATH[/tt] environment prior to being invoked (or as part of it’s startup)

When it was added to UI4, the use of standard [tt]require(“lib”)[/tt] was valid, since the default implementation relied upon one of the [4] standard loaders to pickup/resolve the .lua extension to the Filesystem, but also “manual” amendments to the [tt]package.cpath[/tt] to include right bits… Plugin files could use Lua’s [tt]require[/tt] to include libraries, and in turn those libraries could also use [tt]require[/tt] to include other libraries… all bog standard stuff.

Even the Developer file uploader would NOT compress/LZO any [tt].lua[/tt] file so that the standard Loader could be used.

UI5 has changed this model to compress .lua files AND force developers to use a UI5, and MiOS, specific piece of code to trigger the loading activity (currently the UI5-specifi tag, previously the use of [tt]luupx.require[/tt], and eliminating the module declaration)

The UI5 changes are incompatible with a single-source UI4-UI5 Plugin, and forces plugin authors to either not use .lua files (inlining etc) or to split their codebases…

As I’ve indicated before, this is an unacceptable UI5 upgrade impact.

Anyhow, turns out that by following standard Lua coding patterns, this problem is trivial to fix, in a way that can be both UI4 and UI5 compatible, and let you continue to compress .lua files (in UI5) and not compress them (in UI4)

So the Bug request is simply that you either:

a) Implement a Lua-standard Library Loader (for compressed, and/or crypto’d .lua files) OR;
b) Go back to using non compressed .lua Library files

I like that the Lua files are now consistent with the rest of the files regarding compressing and would like to remain this way.
You don't have to lose that, it's part of the Lua language to let you extend it's Loader array, per the sample code link that I provided in the Bug (and to Aaron)

If done correctly, I should have to add ANY code to my code, since the correct Loader context will be built, and instantiated, by UI5 prior to hitting the first line of my code.

Standard [tt]require[/tt] functions will work with no change to user code if this is done to the Lua-standard model.

Having just upgraded to UI5, I finally understand what @guessed is saying. I’m with him. I do still have to support UI4 for those users who are on Vera1 devices. And I have to support UI5 for at least myself. There’s no way I’m going back to inlining my Lua code in the I_*.xml file.

Not only is @guessed’s suggestion befitting the spirit of Lua, it also happens to work. MCV, I program plugins for you for free because it’s fun. Maintaining two code bases reduces the fun.

AFAIK Aaron already discussed with guessed about this and we should see the results soon.

According to this post, this UI4 → UI5 regression is handled in UI5 Release 1.5.261:
http://forum.micasaverde.com/index.php/topic,9505.msg63305.html#msg63305

NOTE: Just updating post as I’m about to reference it in a bunch of places…

[quote=“guessed, post:2, topic:169892”]

I like that the Lua files are now consistent with the rest of the files regarding compressing and would like to remain this way.

You don’t have to lose that, it’s part of the Lua language to let you extend it’s Loader array, per the sample code link that I provided in the Bug (and to Aaron)[/quote]

It looks like this wasn’t the way compressed loading was implemented. Printing #package.loaders outputs 4, which is the shipped default in vanilla Lua.

I’m trying to have my plugin require() a subsidiary, compressed, Lua file, and it’s not being found. There’s clearly some magic going on in the I_.xml file that lets it require() compressed files, magic that doesn’t apply inside L_.lua(.lzo) files.

Investigating an issue reported by @hugheaves I found than the overloaded require doesn’t work with any compressed Lua file. The compressed modules must be declared with module(), and not using any of the alternative forms.

Is your module declared using the module keyword?

Florin, that was it. I’d just figured it out myself and came to amend my post but you were too quick.