openLuup: Version Log

Release 7.0

Now in the master branch and tagged as v0.7.0.

Special thanks to @amg0 for changes to AltUI to support icon access through port 3480.
Requires build 1233 or later of AltUI.

[ul][li]icons served through port 3480: works remotely (using, eg. weaved.com)[/li]
[li]WSAPI for CGIs - Lua modules can be used as CGIs[/li]
[li]index.html for redirects - [tt]http://openLuupIP:3480/[/tt] redirects to AltUI[/li]
[li][tt]/cmh-lu/[/tt] directory - alternative directory for .xml, .json, and .lua files, etc.[/li]
[li]sundry other bug fixes and improvements[/li][/ul]

Note that the [tt]/www/cmh/skins/default/…[/tt] directory tree is no longer used,
since all icons are served over port 3480 from the [tt]/etc/cmh-ludl/icons/[/tt] directory.
You need to move all your icons to this directory.
There should now be no access through HTTP port 80 at all.

Note also that the documentation has been updated to reflect changes required in installation (less to do now, no port 80 server setup) and more detail on other aspects.

Development Branch

Now updated with a version that can pull AltUI (not yet openLuup) releases from GitHub.
Once I add openLuup updates, this will become the Release 8.0 candidate.

… this release will also handle the licensing of AltUI when not connected to a Vera by the bridge: contact me with your AltUI-registered user name to receive a key.

Release 8 candidate

The development branch has been updated with the Release 8.0 candidate code:

Changes include:

[ul][li]auto-update (at last!), triggered by HTTP request or action in openLuupExtensions[/li]
[li]plugins (AltUI, openLuup itself) loaded from GitHub[/li]
[li]better handling of device / scene / job status[/li]
[li]better compatibility with Windows on PC[/li]
[li]improved AltUI compatibility (thanks @amg0)[/li]
[list]
[li]extended functionality through port_49450 requests and ModifyUserData… [/li]
[li]…support for editing Lua Startup code[/li]
[li]…support for user_data backups initiated from AltUI from Controllers page[/li]
[li]key for licensing of AltUI when not connected to a Vera by the bridge[/li]
[li]logging of AltUI workflow events[/li]
[/list]
[li]HTTP port 3480 server improvements:[/li]
[list]
[li]HTTPS support for luup.inet.wget[/li]
[li]extended mime types, defined in external file[/li]
[li]fix for audio files / Sonos and others (thanks @explorer)[/li]
[/list]
[li]VeraBridge:[/li]
[list]
[li]Include, Exclude, ZWaveOnly filtering (thanks @explorer)[/li]
[li]Mirror openLuup device variables to Vera devices / ‘reverse’ bridging (thanks @logread)[/li]
[li]parent/child hierarchy and invisible devices cloned[/li]
[li]ZWave controller of first bridged Vera is cloned (low-level control of ZWave devices)[/li]
[li]device status updates[/li]
[/list]
[li]updated utilities:[/li]
[list]
[li]openLuup_check - tests for additional CGI files[/li]
[li]openLuup_download - downloads complete version from GitHub[/li]
[/list]
[li]numerous bug fixes[/li][/ul]

Huge thanks to those who have contributed ideas (whether or not listed above), and a special mention to @CudaNet for wrapping some of this into pre-packaged versions for different platforms.

I’m working on some documentation updates, but in the meantime, feedback is welcomed before this gets merged with the master.

Release 8

The master branch has been updated and the new release is tagged as v0.8.0.

Includes the changes described in the Release 8 candidate, but additionally, a working Plugins page for installing and updating selected plugins.

This should be the last version that you have to update manually. (Sadly, it wasn’t.)

[ul][li]auto-update of openLuup, AltUI, VeraBridge from the Plugins page[/li]
[li]plugins (AltUI, openLuup itself) loaded from GitHub[/li]
[li]better handling of device / scene / job status[/li]
[li]better compatibility with Windows on PC[/li]
[li]improved AltUI compatibility (thanks @amg0)[/li]
[li]HTTP port 3480 server improvements[/li]
[li]VeraBridge: Include, Exclude, ZWaveOnly filtering, Mirroring, HouseMode…[/li]
[li]updated utilities[/li]
[li]numerous bug fixes[/li][/ul]

Again, huge thanks to @amg0, @logread, @explorer, @CudaNet, @vosmont and others.

Release 8.1

Mandatory updates to v0.8.0:

[ul][li]fix openLuup destination directory in plugin updates[/li]
[li]fix loader error on empty actions in service files[/li]
[li]fix documentation link for VeraBridge[/li][/ul]

This fixes a serious problem with the destination directory for openLuup updates, which was set to [tt]/etc/cmh-ludl/ [/tt]rather than [tt]/etc/cmh-ludl/openLuup/[/tt]. This needs to be fixed before you can update automatically! The error means that, if you have used it, you will find duplicates of all the files which should be in the [tt]openLuup/[/tt] folder in [tt]/etc/cmh-ludl/[/tt]. To correct this you could simply copy those files into [tt]openLuup/[/tt] replacing the old versions which will still be there.

Alternatively, simply update manually and then future auto-updates should work correctly. Sorry!

A further word on updates:

[ul][li]AltUI updates to the latest version in the master branch[/li]
[li]openLuup and VeraBridge update to the latest version in the development branch.[/li][/ul]

This means that incremental openLuup updates are available without making a tagged release and I’m expecting them to be a bit more frequent (although not up to AltUI’s rate!)

Development Branch: update to VeraBridge

Following this problem statement / suggestion:

[quote=“konradwalsh”]I have a Vera Edge Plus and OpenLuup install.

I want to sync the home modes on each device and looking for suggestions on how to this…[/quote]

and, to be fair, an earlier one here:

I’ve updated the VeraBridge code to mirror either to, or from, a remote Vera’s mode (or not at all.)

The bridge now has a new device variable [tt]HouseModeMirror[/tt], which you can set to one of three values:
[tt]
“0” : no mirroring,
“1” : local mirrors remote machine,
“2” : remote mirrors local machine.
[/tt]
You have to reload to make this change effective. Because of Vera’s built-in delay in changing HouseMode, mode 2 can take a while (typically 30 seconds or more) to kick in, but they should synchronize in the end.

To apply this update, assuming you’re running openLuup v0.8.1, then all you have to do is go the the Plugins page, hit the Update button for VeraBridge, and wait for the reload. Refreshing the browser should show that the VeraBridge version is now “GitHub.development”. The Variables tab of the device should show the new variable (in mode 0) and a Version of “2016.05.23”. Don’t try mirroring HouseMode to/from a UI5 Vera!

Development Branch: Release 8.2 candidate

Significant updates from 8.1:

[ul][li]Plugins Page - this now acts as a sort of mini App Store, showing not just installed plugins, but also others which can be installed just by clicking the Update button. Different versions may be loaded by entering the appropriate name in the Update box (which may be from a master or development branch or a specifically tagged release.) Leaving it empty gets the latest approved version. You can still add other plugins manually by downloading the device files and creating a device.[/li]
[li]Extensions plugin - is now an integrated part of the system. It appears as device #2. It has a number of device variables which monitor system memory and cpu usage and also displays the latest tagged release of openLuup (this may be different from the version you are actually running, which will be displayed, after updating, on the Plugins page.) If you have the old openLuup:Extensions plugin installed, just delete that device. [/li]
[li]VeraBridge - this is now a separate install, enabling independent updates from openLuup itself. This does mean that after an openLuup upgrade you may need to do a VeraBridge upgrade in order to get the latest. The bridge now has the capability to mirror House Mode between openLuup and a remote Vera (three modes: no mirroring, openLuup follows remote Vera, remote Vera follows openLuup.) The bridge also now has an action call which implements the functionality of the previously separate openLuup_getfiles utility, fetching both device files and icons from the bridged Vera and place them into the /files/ and icons/ directories.[/li]
[li]CGI files - no external CGI files or directories are now required for basic system functionality, including Startup Lua editing, user_data file backup, and sysinfo. The CGI folders in /etc/cmh-ludl (namely cgi/ cgi-bin/cmh/ and upnp/control/) are not mandatory, but may exist and contain other WSAPI CGI files, if that’s a configuration you need.[/li]
[li]AltUI - has had a few more tweaks (thanks @amg0) specifically for openLuup. Note, however, that one required file is currently missing from the Plugin page AltUI download, namely dkjson.lua. If you are installing an absolutely new system, you will still need to get this from somewhere (possibly a Vera) I have yet to find a neat way around this. If you’re just updating an existing version, then it will be there already. (install now updated to fix this.)[/li][/ul]

These capabilities should enable a much simpler installation workflow on a bare machine. I just need to document this before the actual release.

Please report any problems before this gets turned into the actual 8.2 release in the master branch.

Release 8.2

[ul][li]Plugins Page - this now acts as a sort of mini App Store, showing not just installed plugins, but also others which can be installed just by clicking the Update button (see attachment.) Different versions may be loaded by entering the appropriate name in the Update box (which may be from a master or development branch or a specifically tagged release.) Leaving it empty gets the latest approved version. You can still add other plugins manually by downloading the device files and creating a device.[/li]
[li]Extensions plugin - is now an integrated part of the system. It appears as device #2 (see attachment.) It has a number of device variables which monitor system memory and cpu usage and also displays the latest tagged release of openLuup (this may be different from the version you are actually running, which will be displayed, after updating, on the Plugins page.) If you have the old openLuup:Extensions plugin installed, just delete that device. [/li]
[li]VeraBridge - this is now a separate install, enabling independent updates from openLuup itself. This does mean that after an openLuup upgrade you may need to do a VeraBridge upgrade in order to get the latest. The bridge now has the capability to mirror House Mode between openLuup and a remote Vera (three modes: no mirroring, openLuup follows remote Vera, remote Vera follows openLuup.) The bridge also now has an action call which implements the functionality of the previously separate openLuup_getfiles utility, fetching both device files and icons from the bridged Vera and placing them into the /files/ and icons/ directories.[/li]
[li]CGI files - no external CGI files or directories are now required for basic system functionality, including Startup Lua editing, user_data file backup, and sysinfo. The CGI folders in /etc/cmh-ludl (namely cgi/ cgi-bin/cmh/ and upnp/control/) are not mandatory, but may exist and contain other WSAPI CGI files, if that’s a configuration you need.[/li]
[li]AltUI - has had a few more tweaks (thanks @amg0) specifically for openLuup. openLuup also now has support for AltUI scene updates via HTTP POST requests.[/li]
[li]DataYours - DataYours installs itself as an AltUI Data Storage Provider, creates a local database directory “/etc/cmh-ludl/whisper/”, and starts logging CPU and memory usage every 5 minutes for the last week (see attachment.)[/ul]

These capabilities enable a much simpler installation workflow on a bare machine.

Release 8.4 (just don’t ask about 8.3)

Full integration of AltAppStore.

Suggested upgrade sequence, assuming you are already running v0.8.2 (otherwise you’ll have to download and install manually, or use the latest Utilities/openLuup_install.lua):

[ol][li]VeraBridge (entering into the Plugins page Update box: master) - contains a required fix to avoid failure of AltAppStore plugin![/li]
[li]AltUI (using the usual page refresh method) - ensure you’re at version 1778 or more[/li]
[li]openLuup (entering into the Plugins page Update box: master) - to finally install the latest[/li][/ol]

A final browser page refresh should have you up and running…

[ul][li]The AltAppStore plugin is an integral part of this and future releases, so you shouldn’t ever need to update it from the App Store.[/li]
[li]that plugin now forces a luup.reload() after every successful update.[/li]
[li]other installed plugins may either be updated from the Plugins page, or from the AppStore.[/li][/ul]

The new Utilities/openLuup-install.lua file is all you need (after having installed Lua and its libraries) to get to a running openLuup system from scratch.

Development Branch: Release 8.5 candidate

Significant updates include:

[ul][li]POST requests of unlimited length[/li]
[li]HTTP server and WSAPI refactoring - prerequisite for planned asynchronous VeraBridge (faster)[/li]
[li]LZAP data compression for user_data backups[/li]
[li]HTML backup page with file Download option[/li]
[li]openLuup_reload script accepts compressed user_data files[/li]
[li]chdev.sync() change for RaZberry plugin[/li][/ul]

Simply installed by typing development into the Update box on the Plugins page and clicking the update button.

Refactoring of the HTTP server has been significant. I would be very grateful of any feedback on this, particularly relating to timeouts on large file transfers (which was a problem once, but I hope has not resurfaced.)

2016 Release 8.5 (v16.8.5)

Further refactoring, but pretty much the same as the 8.5 candidate release.

Simply installed by clicking the openLuup Update button on the Plugins page.

Development Branch: 2016 Release 10.4

[ul][li]added Apache 2.0 license[/li]
[li]Grafana support infrastructure (with DataYours installed)[/li][/ul]

This allows the use of Grafana as a dashboard for any DataYours (and/or dataMine) databases, whether being used as AltUI Data Storage Providers, or not. You do need to have the latest DataYours (16.10.4 or later) installed for this to work. Configure Grafana to have a Graphite database server at [tt]http:openLuupIP:3480[/tt].

The file graphite_cgi.lua needs to be copied from the DataYours GitHub to /etc/cmh-ludl/. At the moment, the automatic install doesn’t do that. Sorry.

To be migrated to the master branch if no bad things happen.

Development Branch: 2016 Release 11.26

This development release incorporates many of the features which have been in testing for a while:

[ul][li]updated documentation (particularly for new system installs)[/li]
[li]AltAppStore - improved error checking[/li]
[li]VeraBridge - reducing polling overhead for status requests[/li]
[li]graphite_cgi - CGI interface to DataYours now bundled with openLuup system (for Grafana, etc.)[/li]
[li]luup.io - add implementation for io.read() with “crlf” protocol[/li]
[li]status requests - startup jobs now shown correctly[/li]
[li]scenes - day-of-week and day-of-month timing error fixed, and status updates[/li]
[li]openLuup plugin room allocation preserved across reload (bugfix)[/li]
[li]extended and updated unit tests[/li]
[li]miscellaneous bug fixes and improvements[/li][/ul]

Simply update to this version on the plugin page using development in the Update box

This is a release candidate for the next master branch update.

Development Branch: 2016 Release 12.10

Significant changes include the relocation of logs files to /etc/cmh-ludl/logs/and a change to the default user_data.json checkpoint frequency. Some system internals may be inspected using the new console viewer UI (http://openLuupIP:3480/console.)

[hr]

Log files

openLuup now generates an extra log file LuaUPnP_startup.log, which is written to /etc/cmh-ludl/logs/ and is versioned to save the previous five reloads. This is a very convenient way to check the last reload times of a system. Typically very short, this log finishes after Lua Startup code has been run, and then…

…logging continues with a file in a location of your choice (default is logs/), limited to a number of lines you specify, and versioned to a set number. Could be anywhere in the mounted file system.

user_data.json

This is generally written to /etc/cmh-ludl/ (don’t change this) and is checkpointed after the first 6 minutes of a system reload and thereafter every 60 minutes.

[hr]

All the above parameters may be set in the Lua Startup code, thus:

-- openLuup configuration options:
luup.attr_set ("openLuup.Logfile.Name", "logs/LuaUPnP.log")   -- full path to log file
luup.attr_set ("openLuup.Logfile.Lines", 2000) 
luup.attr_set ("openLuup.Logfile.Versions", 5)
luup.attr_set ("openLuup.UserData.Checkpoint", 60)      -- checkpoint every 60 minutes

A new console interface [b]http://openLuupIP:3480/console[/b] allows easy perusal of these (and other) system parameters. There is a link to this from the openLuup device panel.

2016 Release 12.21

Master branch updated from development. Significant changes since last tagged release (v16.8.5):

[ul][li]added Apache 2.0 license[/li]
[li]updated documentation (particularly for new system installs)[/li]
[li]AltAppStore - improved error checking[/li]
[li]VeraBridge - reducing polling overhead for status requests[/li]
[li]graphite_cgi - CGI interface to DataYours now bundled with openLuup system (for Grafana, etc.)[/li]
[li]relocation of logs files to /etc/cmh-ludl/logs/[/li]
[li]change to the default user_data.json checkpoint frequency (to 60 minutes)[/li]
[li]new console UI to view some system internals (scheduler, parameters, backups, logs)[/li]
[li]luup.io - add implementation for io.read() with “crlf” protocol[/li]
[li]status requests - startup jobs now shown correctly[/li]
[li]scenes - day-of-week and day-of-month timing error fixed, and status updates[/li]
[li]openLuup plugin room allocation preserved across reload[/li]
[li]extended and updated unit tests[/li]
[li]miscellaneous bug fixes and improvements[/li][/ul]

Also tagged as v16.12.21

Development Branch: 2017 Release 1.18

The openLuup plugin now has a HouseMode variable, which may be used as a variable watch trigger to kick off scenes when the house mode is changed (no need for a separate HouseMode plugin.)

Development Branch: 2017 Release 2.8

Includes a required fix for openLuup to handle some POST requests from AltUI (thanks @amg0)

2017 Release 3.9

Master branch roll-up of recent developments:

[ul][li]HouseMode variable on openLuup device - can trigger scenes (no need for HouseMode plugin)[/li]
[li]server allows POST requests (used now by AltUI for some actions)[/li]
[li]VeraBridge now registers with AltUI as a Data Storage Provider for mirroring variables to Vera[/li]
[li]VeraBridge BridgeScenes variable - enable/disable linking to remote scenes[/li]
[li]new storage schemas for DataYours storage provider (1/5/10/20 min, 1/3/6 hour, 1 day sample rates)[/li]
[li]updated documentation[/li]
[li]misc. bug fixes[/li][/ul]

Also tagged as v17.3.9

Development Branch: 2017 Release 3.15

[ul][li]added server configuration parameters to openLuup attribute[/li]
[li]changed server backlog default to 2000 (thanks @explorer)[/li][/ul]

The above changes add user configurability to the port 3480 server and improve performance in complex systems. (Change these at your own risk!)

2018 Release 2.26

Master branch roll-up of developments from over the last year (it’s been too long!)

The changes are too extensive to document in detail, but recent updates have been to anticipate Vera security ‘improvements’ to be rolled in next month and also the recent GitHub changes which deprecate old SSL security access levels. As a result, the normal update approach (simply clicking the update ‘refresh’ button on the Plugins page) will not work and you will need to use the following procedure to allow the AltAppStore plugin to work and then update openLuup as usual:

openLuup Fix only for AltAppStore / GitHub SSL error

Changes in this update include:

  • server - faster CGI and file response
  • timers - sunrise/set accuracy, noon equinox date error fixed
  • scenes: global prolog/epilog code options, and individual scene finalisers
  • logs - fixed error leading to loss of AltUI variable history for scenes
  • requests - implement invoke request
  • luup.ir module - implemented
  • AltAppStore plugin - update SSL security to tls v1.2
  • VeraBridge plugin- remove old variable mirroring method, display house mode, temporary scene copies
  • openLuup plugin - add HouseMode variable for local machine
  • updated documentation
  • misc. bug fixes

Also tagged as v18.2.26