openLuup: Version Log

Release 5.4

[ul][li]fix scene save issues[/li]
[li]ShutdownCode attribute - user code to run at shutdown and reload[/li]
[li]extended unit testing[/li][/ul]

The ShutdownCode attribute can be set with luup.attr_set in this way:

luup.attr_set ("ShutdownCode", [[
  -- anything you like here, really
  luup.log "hello from user shutdown code!"
]]

For a specific (and useful) application of this, see the openLuup Nest thread.

Release 5.5

[ul][li]refactor VeraBridge[/li]
[li]openLuup Extensions plugin[/li]
[li]create_device action call[/li]
[li]refactor luup.chdev module[/li]
[li]fix server query/file logic (thanks @vosmont)[/li]
[li]bug fixes[/li][/ul]

Some important points to note:

[ul][li]The VeraBridge is NOT compatible with the old version, you will have to rebuild from a startup.lua file rather than an existing user_data.json (sorry)[/li]
[li]There is a new plugin openLuup:Extensions.[/li][/ul]

VeraBridge

There was an intrinsic problem with the way that the bridge used to treat device numbers: any new device, be it local or cloned from a remote machine, would receive the next available deviceID, as in a standard Vera. This meant that any change to local devices or ones on remote machines (ie. creation/deletion) could result in an extensive renumbering of all devices in openLuup
 clearly not acceptable.

The approach now is for local device numbers to work as in Vera - a new device gets the next available ID. However, cloned devices from remote machines get device IDs allocated from higher-numbered blocks. The first VeraBridge uses device numbers starting at 10,000, the second one at 20,000, and so on. There is also some consistency in the remote device numbering in that the original Vera deviceIDs are simply shifted by the bridge offset. Thus device 42 on your first bridged Vera will always be device 10042 on openLuup.

Local and remote devices are thus completely isolated in terms of device numbering interactions. This will make scene triggers and actions much more stable.

openLuupExtensions

This is a new plugin - move the files to /etc/cmh-ludl/ and install with:

luup.create_device ('', '', nil, "D_openLuupExtensions.xml")


 or use the +Create button on the AltUI interface (which works now! - thanks @amg0)

The philosophy here is that openLuup itself should remain as true as possible* to standard Vera functionality. That way we can ensure that most plugins written for Vera run also under openLuup. HOWEVER
 now having gained the freedom to make improvements, the Extensions plugin is going to add extra facilities over time.

At the moment, it does very little:

[ul][li]front panel display shows system uptime and memory usage[/li]
[li]similar data can be read from device variables (and used as device variable watch triggers or pushed to ThingSpeak)[/li]
[li]a few actions have been implemented, more will come. At the moment you can set StartupCode and ShutdownCode[/li][/ul]

A feature to come soon (hopefully) is the ability to upgrade openLuup itself from the GitHub repository in much the same way as the AltUI update works now.


  • with the notable exceptions of: not crashing, not running out of memory, and not using much CPU.

Release 6.0

[ul][li]improved integration with the ALTUI interface[/li]
[li]uses the LuaFileSystem* (lfs) module to enhance portability (runs well on Windows)[/li]
[li]device attributes better preserved over reloads[/li]
[li]lul_device global in plugins[/li]
[li]nil device and variable parameters supported in luup.variable_watch[/li]
[li]nil device parameter supported in luup.variable_set/get[/li]
[li]VeraBridge plugin now clones remote scenes[/li]
[li]considerable internal refactoring to support future enhancements[/li][/ul]

Important point to note:

[ul][li]LuaFileSystem* is already required by some plugins, but if not installed you will need to do so.[/li]
[li]Handling of altids for remote Zwave devices is different, you will have to rebuild from a startup.lua file rather than an existing user_data.json (sorry)[/li]
[li]Both the VeraBridge and openLuupExtensions plugin are distributed in sub-directories, the contents of which need to be moved to /etc/cmh-ludl/ for correct installation.[/li]
[li]You have to run the latest AltUI for scenes to work correctly - V 0.98.895 or later.[/ul]

VeraBridge

The bridge now clones remote scenes. Huge thanks to @amg0 to make AltUI amenable to this trick. In the same way that remote devices are allocated ids in the 10,000 range, and above, so it is with remote scenes. However, locally defined scenes still retain low numbers. In fact, a newly defined scene will have the lowest empty slot in the luup.scenes table.

You can’t see the actions, or triggers, or timers, of the remote scene, but you can manually trigger it. If you want to, you can also add local variable watch triggers and timers to the scene. You won’t be able to see or edit these from the remote Vera (it just receives a command to run.)

This may (or may not) assist integration and transfer of logic from Vera to openLuup.

This release is in a new branch of the GitHub repository: https://github.com/akbooer/openLuup/tree/Release-6, and also tagged as v0.6.0 (ie. Version 0, Release 6.0) It has undergone extensive testing on Mac, Windows, Debian, and Open-WRT, but there is, I suppose, the slightest possibility of some new bugs.

Release 6.2 (just don’t ask about 6.1)

[ul][li]more extensive use of LuaFileSystem library to improve portability[/li]
[li]new http.lua module to fix a bug in LuaSocket[/li]
[li]VeraBridge updates[/li]
[li]Extensions plugin update[/li]
[li]support for AltUI donation ‘licensing’[/li][/ul]

Important points to note:

The imminent version 1.0 of AltUI has a donation ‘licensing’ scheme to remove the donation tag for those who have contributed in one way or another. This version of openLuup supports that mechanism for configurations which use VeraBridge to link to a real Vera with a primary account that is recognised by AltUI. If you are using openLuup in a configuration without a link to such a Vera (eg. you’re only using it as a MySensors controller, or the like) then you’ll have to ask me for a userId if you want to remove the donation reminder.

Do remember to move the files in the openLuupExtensions and Utilities and VeraBridge folders into your main /etc/cmh-ludl/ directory, along with the new openLuup folder.

I strongly recommend you backup your user_data.json file prior to this (or any) upgrade. It should handle your existing file, but may perform a couple of reloads for the VeraBridge. In the event of needing to start from a startup.lua file again, your scenes can be restored from the old file (manually at the moment, I need to automate this.)

Release 6.3

After extensive testing, mostly by @cybrmage, I have updated the GitHub code master with the latest io module.

Changes include:

[ul][li]correctly functioning luup.io.intercept()[/li]
[li]io protocol implementations for raw, cr, crlf, (but not stxetx)[/li]
[li]correctly functioning timeouts in io.read()[/li]
[li]device number and tcp socket shown in log entries[/li]
[li]devices now have ‘disabled’ attribute[/li][/ul]

Release 7.0 candidate

The development branch has been updated with the Release 7.0 candidate code:

Changes include:

[ul][li]icons served through port 3480: remote ready![/li]
[li]WSAPI for CGIs - Lua modules can be used as CGIs[/li]
[li]index.html for redirects - [tt]http://openLuupIP:3480/[/tt] redirects to AltUI[/li]
[li][tt]/cmh-lu/[/tt] directory - alternative directory for .xml, .json, and .lua files, etc.[/li]
[li]sundry other bug fixes and improvements[/li][/ul]

Note that the [tt]/www/cmh/skins/default/
[/tt] directory tree is no longer used, since all icons are served over port 3480 from the [tt]/etc/cmh-ludl/icons/[/tt] directory. In fact, there should now be no access through HTTP port 80 at all.

Note also that the documentation has been updated to reflected changes required in installation (less to do now) and more detail on other aspects.

Huge thanks to those who have contributed ideas, testing, and sometimes even a bit of code (you know who you are!)

Feedback welcomed before this gets merged with the master.

Release 7.0

Now in the master branch and tagged as v0.7.0.

Special thanks to @amg0 for changes to AltUI to support icon access through port 3480.
Requires build 1233 or later of AltUI.

[ul][li]icons served through port 3480: works remotely (using, eg. weaved.com)[/li]
[li]WSAPI for CGIs - Lua modules can be used as CGIs[/li]
[li]index.html for redirects - [tt]http://openLuupIP:3480/[/tt] redirects to AltUI[/li]
[li][tt]/cmh-lu/[/tt] directory - alternative directory for .xml, .json, and .lua files, etc.[/li]
[li]sundry other bug fixes and improvements[/li][/ul]

Note that the [tt]/www/cmh/skins/default/
[/tt] directory tree is no longer used,
since all icons are served over port 3480 from the [tt]/etc/cmh-ludl/icons/[/tt] directory.
You need to move all your icons to this directory.
There should now be no access through HTTP port 80 at all.

Note also that the documentation has been updated to reflect changes required in installation (less to do now, no port 80 server setup) and more detail on other aspects.

Development Branch

Now updated with a version that can pull AltUI (not yet openLuup) releases from GitHub.
Once I add openLuup updates, this will become the Release 8.0 candidate.


 this release will also handle the licensing of AltUI when not connected to a Vera by the bridge: contact me with your AltUI-registered user name to receive a key.

Release 8 candidate

The development branch has been updated with the Release 8.0 candidate code:

Changes include:

[ul][li]auto-update (at last!), triggered by HTTP request or action in openLuupExtensions[/li]
[li]plugins (AltUI, openLuup itself) loaded from GitHub[/li]
[li]better handling of device / scene / job status[/li]
[li]better compatibility with Windows on PC[/li]
[li]improved AltUI compatibility (thanks @amg0)[/li]
[list]
[li]extended functionality through port_49450 requests and ModifyUserData
 [/li]
[li]
support for editing Lua Startup code[/li]
[li]
support for user_data backups initiated from AltUI from Controllers page[/li]
[li]key for licensing of AltUI when not connected to a Vera by the bridge[/li]
[li]logging of AltUI workflow events[/li]
[/list]
[li]HTTP port 3480 server improvements:[/li]
[list]
[li]HTTPS support for luup.inet.wget[/li]
[li]extended mime types, defined in external file[/li]
[li]fix for audio files / Sonos and others (thanks @explorer)[/li]
[/list]
[li]VeraBridge:[/li]
[list]
[li]Include, Exclude, ZWaveOnly filtering (thanks @explorer)[/li]
[li]Mirror openLuup device variables to Vera devices / ‘reverse’ bridging (thanks @logread)[/li]
[li]parent/child hierarchy and invisible devices cloned[/li]
[li]ZWave controller of first bridged Vera is cloned (low-level control of ZWave devices)[/li]
[li]device status updates[/li]
[/list]
[li]updated utilities:[/li]
[list]
[li]openLuup_check - tests for additional CGI files[/li]
[li]openLuup_download - downloads complete version from GitHub[/li]
[/list]
[li]numerous bug fixes[/li][/ul]

Huge thanks to those who have contributed ideas (whether or not listed above), and a special mention to @CudaNet for wrapping some of this into pre-packaged versions for different platforms.

I’m working on some documentation updates, but in the meantime, feedback is welcomed before this gets merged with the master.

Release 8

The master branch has been updated and the new release is tagged as v0.8.0.

Includes the changes described in the Release 8 candidate, but additionally, a working Plugins page for installing and updating selected plugins.

This should be the last version that you have to update manually. (Sadly, it wasn’t.)

[ul][li]auto-update of openLuup, AltUI, VeraBridge from the Plugins page[/li]
[li]plugins (AltUI, openLuup itself) loaded from GitHub[/li]
[li]better handling of device / scene / job status[/li]
[li]better compatibility with Windows on PC[/li]
[li]improved AltUI compatibility (thanks @amg0)[/li]
[li]HTTP port 3480 server improvements[/li]
[li]VeraBridge: Include, Exclude, ZWaveOnly filtering, Mirroring, HouseMode
[/li]
[li]updated utilities[/li]
[li]numerous bug fixes[/li][/ul]

Again, huge thanks to @amg0, @logread, @explorer, @CudaNet, @vosmont and others.

Release 8.1

Mandatory updates to v0.8.0:

[ul][li]fix openLuup destination directory in plugin updates[/li]
[li]fix loader error on empty actions in service files[/li]
[li]fix documentation link for VeraBridge[/li][/ul]

This fixes a serious problem with the destination directory for openLuup updates, which was set to [tt]/etc/cmh-ludl/ [/tt]rather than [tt]/etc/cmh-ludl/openLuup/[/tt]. This needs to be fixed before you can update automatically! The error means that, if you have used it, you will find duplicates of all the files which should be in the [tt]openLuup/[/tt] folder in [tt]/etc/cmh-ludl/[/tt]. To correct this you could simply copy those files into [tt]openLuup/[/tt] replacing the old versions which will still be there.

Alternatively, simply update manually and then future auto-updates should work correctly. Sorry!

A further word on updates:

[ul][li]AltUI updates to the latest version in the master branch[/li]
[li]openLuup and VeraBridge update to the latest version in the development branch.[/li][/ul]

This means that incremental openLuup updates are available without making a tagged release and I’m expecting them to be a bit more frequent (although not up to AltUI’s rate!)

Development Branch: update to VeraBridge

Following this problem statement / suggestion:

[quote=“konradwalsh”]I have a Vera Edge Plus and OpenLuup install.

I want to sync the home modes on each device and looking for suggestions on how to this
[/quote]

and, to be fair, an earlier one here:

I’ve updated the VeraBridge code to mirror either to, or from, a remote Vera’s mode (or not at all.)

The bridge now has a new device variable [tt]HouseModeMirror[/tt], which you can set to one of three values:
[tt]
“0” : no mirroring,
“1” : local mirrors remote machine,
“2” : remote mirrors local machine.
[/tt]
You have to reload to make this change effective. Because of Vera’s built-in delay in changing HouseMode, mode 2 can take a while (typically 30 seconds or more) to kick in, but they should synchronize in the end.

To apply this update, assuming you’re running openLuup v0.8.1, then all you have to do is go the the Plugins page, hit the Update button for VeraBridge, and wait for the reload. Refreshing the browser should show that the VeraBridge version is now “GitHub.development”. The Variables tab of the device should show the new variable (in mode 0) and a Version of “2016.05.23”. Don’t try mirroring HouseMode to/from a UI5 Vera!

Development Branch: Release 8.2 candidate

Significant updates from 8.1:

[ul][li]Plugins Page - this now acts as a sort of mini App Store, showing not just installed plugins, but also others which can be installed just by clicking the Update button. Different versions may be loaded by entering the appropriate name in the Update box (which may be from a master or development branch or a specifically tagged release.) Leaving it empty gets the latest approved version. You can still add other plugins manually by downloading the device files and creating a device.[/li]
[li]Extensions plugin - is now an integrated part of the system. It appears as device #2. It has a number of device variables which monitor system memory and cpu usage and also displays the latest tagged release of openLuup (this may be different from the version you are actually running, which will be displayed, after updating, on the Plugins page.) If you have the old openLuup:Extensions plugin installed, just delete that device. [/li]
[li]VeraBridge - this is now a separate install, enabling independent updates from openLuup itself. This does mean that after an openLuup upgrade you may need to do a VeraBridge upgrade in order to get the latest. The bridge now has the capability to mirror House Mode between openLuup and a remote Vera (three modes: no mirroring, openLuup follows remote Vera, remote Vera follows openLuup.) The bridge also now has an action call which implements the functionality of the previously separate openLuup_getfiles utility, fetching both device files and icons from the bridged Vera and place them into the /files/ and icons/ directories.[/li]
[li]CGI files - no external CGI files or directories are now required for basic system functionality, including Startup Lua editing, user_data file backup, and sysinfo. The CGI folders in /etc/cmh-ludl (namely cgi/ cgi-bin/cmh/ and upnp/control/) are not mandatory, but may exist and contain other WSAPI CGI files, if that’s a configuration you need.[/li]
[li]AltUI - has had a few more tweaks (thanks @amg0) specifically for openLuup. Note, however, that one required file is currently missing from the Plugin page AltUI download, namely dkjson.lua. If you are installing an absolutely new system, you will still need to get this from somewhere (possibly a Vera) I have yet to find a neat way around this. If you’re just updating an existing version, then it will be there already. (install now updated to fix this.)[/li][/ul]

These capabilities should enable a much simpler installation workflow on a bare machine. I just need to document this before the actual release.

Please report any problems before this gets turned into the actual 8.2 release in the master branch.

Release 8.2

[ul][li]Plugins Page - this now acts as a sort of mini App Store, showing not just installed plugins, but also others which can be installed just by clicking the Update button (see attachment.) Different versions may be loaded by entering the appropriate name in the Update box (which may be from a master or development branch or a specifically tagged release.) Leaving it empty gets the latest approved version. You can still add other plugins manually by downloading the device files and creating a device.[/li]
[li]Extensions plugin - is now an integrated part of the system. It appears as device #2 (see attachment.) It has a number of device variables which monitor system memory and cpu usage and also displays the latest tagged release of openLuup (this may be different from the version you are actually running, which will be displayed, after updating, on the Plugins page.) If you have the old openLuup:Extensions plugin installed, just delete that device. [/li]
[li]VeraBridge - this is now a separate install, enabling independent updates from openLuup itself. This does mean that after an openLuup upgrade you may need to do a VeraBridge upgrade in order to get the latest. The bridge now has the capability to mirror House Mode between openLuup and a remote Vera (three modes: no mirroring, openLuup follows remote Vera, remote Vera follows openLuup.) The bridge also now has an action call which implements the functionality of the previously separate openLuup_getfiles utility, fetching both device files and icons from the bridged Vera and placing them into the /files/ and icons/ directories.[/li]
[li]CGI files - no external CGI files or directories are now required for basic system functionality, including Startup Lua editing, user_data file backup, and sysinfo. The CGI folders in /etc/cmh-ludl (namely cgi/ cgi-bin/cmh/ and upnp/control/) are not mandatory, but may exist and contain other WSAPI CGI files, if that’s a configuration you need.[/li]
[li]AltUI - has had a few more tweaks (thanks @amg0) specifically for openLuup. openLuup also now has support for AltUI scene updates via HTTP POST requests.[/li]
[li]DataYours - DataYours installs itself as an AltUI Data Storage Provider, creates a local database directory “/etc/cmh-ludl/whisper/”, and starts logging CPU and memory usage every 5 minutes for the last week (see attachment.)[/ul]

These capabilities enable a much simpler installation workflow on a bare machine.

Release 8.4 (just don’t ask about 8.3)

Full integration of AltAppStore.

Suggested upgrade sequence, assuming you are already running v0.8.2 (otherwise you’ll have to download and install manually, or use the latest Utilities/openLuup_install.lua):

[ol][li]VeraBridge (entering into the Plugins page Update box: master) - contains a required fix to avoid failure of AltAppStore plugin![/li]
[li]AltUI (using the usual page refresh method) - ensure you’re at version 1778 or more[/li]
[li]openLuup (entering into the Plugins page Update box: master) - to finally install the latest[/li][/ol]

A final browser page refresh should have you up and running


[ul][li]The AltAppStore plugin is an integral part of this and future releases, so you shouldn’t ever need to update it from the App Store.[/li]
[li]that plugin now forces a luup.reload() after every successful update.[/li]
[li]other installed plugins may either be updated from the Plugins page, or from the AppStore.[/li][/ul]

The new Utilities/openLuup-install.lua file is all you need (after having installed Lua and its libraries) to get to a running openLuup system from scratch.

Development Branch: Release 8.5 candidate

Significant updates include:

[ul][li]POST requests of unlimited length[/li]
[li]HTTP server and WSAPI refactoring - prerequisite for planned asynchronous VeraBridge (faster)[/li]
[li]LZAP data compression for user_data backups[/li]
[li]HTML backup page with file Download option[/li]
[li]openLuup_reload script accepts compressed user_data files[/li]
[li]chdev.sync() change for RaZberry plugin[/li][/ul]

Simply installed by typing development into the Update box on the Plugins page and clicking the update button.

Refactoring of the HTTP server has been significant. I would be very grateful of any feedback on this, particularly relating to timeouts on large file transfers (which was a problem once, but I hope has not resurfaced.)

2016 Release 8.5 (v16.8.5)

Further refactoring, but pretty much the same as the 8.5 candidate release.

Simply installed by clicking the openLuup Update button on the Plugins page.

Development Branch: 2016 Release 10.4

[ul][li]added Apache 2.0 license[/li]
[li]Grafana support infrastructure (with DataYours installed)[/li][/ul]

This allows the use of Grafana as a dashboard for any DataYours (and/or dataMine) databases, whether being used as AltUI Data Storage Providers, or not. You do need to have the latest DataYours (16.10.4 or later) installed for this to work. Configure Grafana to have a Graphite database server at [tt]http:openLuupIP:3480[/tt].

The file graphite_cgi.lua needs to be copied from the DataYours GitHub to /etc/cmh-ludl/. At the moment, the automatic install doesn’t do that. Sorry.

To be migrated to the master branch if no bad things happen.

Development Branch: 2016 Release 11.26

This development release incorporates many of the features which have been in testing for a while:

[ul][li]updated documentation (particularly for new system installs)[/li]
[li]AltAppStore - improved error checking[/li]
[li]VeraBridge - reducing polling overhead for status requests[/li]
[li]graphite_cgi - CGI interface to DataYours now bundled with openLuup system (for Grafana, etc.)[/li]
[li]luup.io - add implementation for io.read() with “crlf” protocol[/li]
[li]status requests - startup jobs now shown correctly[/li]
[li]scenes - day-of-week and day-of-month timing error fixed, and status updates[/li]
[li]openLuup plugin room allocation preserved across reload (bugfix)[/li]
[li]extended and updated unit tests[/li]
[li]miscellaneous bug fixes and improvements[/li][/ul]

Simply update to this version on the plugin page using development in the Update box

This is a release candidate for the next master branch update.

Development Branch: 2016 Release 12.10

Significant changes include the relocation of logs files to /etc/cmh-ludl/logs/and a change to the default user_data.json checkpoint frequency. Some system internals may be inspected using the new console viewer UI (http://openLuupIP:3480/console.)

[hr]

Log files

openLuup now generates an extra log file LuaUPnP_startup.log, which is written to /etc/cmh-ludl/logs/ and is versioned to save the previous five reloads. This is a very convenient way to check the last reload times of a system. Typically very short, this log finishes after Lua Startup code has been run, and then



logging continues with a file in a location of your choice (default is logs/), limited to a number of lines you specify, and versioned to a set number. Could be anywhere in the mounted file system.

user_data.json

This is generally written to /etc/cmh-ludl/ (don’t change this) and is checkpointed after the first 6 minutes of a system reload and thereafter every 60 minutes.

[hr]

All the above parameters may be set in the Lua Startup code, thus:

-- openLuup configuration options:
luup.attr_set ("openLuup.Logfile.Name", "logs/LuaUPnP.log")   -- full path to log file
luup.attr_set ("openLuup.Logfile.Lines", 2000) 
luup.attr_set ("openLuup.Logfile.Versions", 5)
luup.attr_set ("openLuup.UserData.Checkpoint", 60)      -- checkpoint every 60 minutes

A new console interface [b]http://openLuupIP:3480/console[/b] allows easy perusal of these (and other) system parameters. There is a link to this from the openLuup device panel.