openLuup on Docker (Hub)

[vwout]

To simplify plugin development I created a Docker image for openLuup and made it available on Docker Hub.

The Docker image is based on Debian (9, slim) and installs the master branch release of openLuup.
The image uses the automated installation script of openLuup and therefore also contains AltUI from the start.

It can be used directly from Docker Hub using

docker pull vwout/openluup

The image exposes port 3480, the (default) port that runs the openLuup console and AltUI.

To allow settings and installed plugins to survive removal of the container, the image uses a number of volumes:

[ul][li]The openluup environment (/etc/cmh-ludl/)[/li]
[li]Logs (/etc/cmh-ludl/logs)[/li]
[li]Backups (/etc/cmh-ludl/backup)[/li][/ul]

The directory /etc/cmh-lu is supported to be mounted as a bind for easy plugin development.

More details on how to use the image are available on Github.
The Github repository also contains a docker-compose file that allows quick creation of your own image based on either the master or development branch. The docker-compose setup also defines named volumes that are automatically mounted to an openLuup container.

The sources for the image are available on GitHub.

[akbooer]

Kudos to @vwout for creating this Docker image.

This has been requested several times in the past, so I hope that we get some willing folk to try it out.

[RHCPNG]

This is really great, vwout! Thanks! I will certainly try it in the future.

[vwout]

The openLuup Docker image was updated to Master Branch: openLuup 2018 Release 8.10.

The scripts for generating the image received some fixes and the image now is about 16MB smaller because of a small optimization during the build.
For details, check the Github repository at vwout/docker-openluup.

[Stuart]

@vwout –

Fantastic! I’m about to start using openluup and recently changed by basement server to Centos with the intention of doing most everything using docker. It was previously esxi but I wanted to trim down a bit on the vm overheads, fool around with docker etc since it’s a small server.

Which brings me to my point – I’m wondering if maybe openluup would be viable on a smaller *nix install in docker versus Debian? Put another way - what is the minimum set of *nix capabilities that openLuup needs. I’m guessing the Debian base is quite “beefy” (~125MB) compared to (say) Alpine at about 5MB.

Thoughts anyone ?

[akbooer]

Which brings me to my point – I’m wondering if maybe openluup would be viable on a smaller *nix install in docker versus Debian? Put another way - what is the minimum set of *nix capabilities that openLuup needs.

I know nothing about Docker, which is why I’m grateful to @vwout for this thread.

However, in terms of required OS resources, openLuup needs nothing but its interpreter and a few standard Lua libraries. I’ve even had it running on an old Arduino Yun board (not on the Arduino, obviously, but the other single chip system running a stripped-down OpenWrt install.) There may be a few basic shell utilities which some plugins use with os.execute() - naughty them!

[Stuart]

However, in terms of required OS resources, openLuup needs nothing but its interpreter and a few standard Lua libraries.

I too am grateful - I hope nothing I said seemed to be negative. When I saw this docker offering, I immediately installed it !!

The thing with docker is that the containers require an operating system (so in effect you have an operating system inside the docker container that is deployed on the host operating system). Sounds like overhead (which it is) but it also has many advantages because of the isolation.

The size does not matter too much, given sufficient resource on the host, so my interest was more a nicety than anything too serious. Hence my questions about the requirements of openluup. From what you have said it may be that a smaller footprint is possible. I know my plugin uses several os.execute calls, all basic with the exception of openssl

If I get some time over the next couple of weeks I’ll do some experiments. First though is cleaning up GCal3 and github

[vwout]

Thanks for the feedback and heads up.

I deliberately choose Debian as a base. On one hand it is easier to setup, since you can e.g. just apt-get lua. On the other hand, a Debian image has more out of the box features compared to something like alpine, which makes the image easier to use by others that are more used to running Openluup on regular Linux.

A bit on Docker in general: To reduce the overhead, I choose the slim base image of Debian Stretch. The slim image is (only) 55MB. Keep in mind that Docker images use layers. If you use a number of services (images), it actually is likely that you have more images that are based on Debian. In that case, the base image is only present once on your system, not in copies for every image. And regarding the overhead, please note that you don’t have a full OS, with its services and stuff, running in the container. Unless you spin it, no other processes besides the one you start, is running in a container.

The complete openluup image is 72.7MB in its current state.
But I do get your point.

Making a small image with lua does not seem to be that easy by judging several images that are around, all based on alpine:

• pirogoeth/alpine-lua: 1K pulls, last updated 2 years ago: 176MB
• superpaintman/lua: 10K+ pulls, last updated 2 years ago: 612MB

Though there is light at the end of the tunnel: akorn/luarocks: 12MB

[Stuart]

Making a small image with lua does not seem to be that easy by judging several images that are around, all based on alpine:

Makes sense.

Since I like to use docker-compose, I thought I’d post the docker-compose.yml file I use as a starting point for others who my be inclined.
I include a mount point to use for transferring files into and out of the running container (via /mnt/uploads in the container).

version: '2'
services:
  openluup:
    container_name: openluup
    image: vwout/openluup
    restart:  unless-stopped 
    environment:
      - TZ=America/Denver
    network_mode: host
    volumes:
      - <a convenient folder on your host>:/mnt/uploads

[vwout]

Thanks for sharing the compose file.
There is one in my github repository that hosts the sources for my docker image as well, but that focusses more on building the various variants than on using the ready-made image from docker hub.

Even though I have some doubts on the effective added value, your request for a smaller image challenged me. Since I investigated it already a bit, I knew it was possible. So I created an additional variant of the openLuup image based on Alpine. And indeed, the image is a lot smaller. Total size including everything is 18MB now 8)
The Dockerfile is more complex, but as a user that should not bother you too much.

The source is on github and the image vwout/openluup:alpine is on Docker Hub. Be sure to use the tag [font=courier]alpine[/font].
I must admit that I did not spend a lot of time on testing this variant, so please provide feedback.

[akbooer]

Oh, nice work!

The 18Mb number fits much more comfortably with the whole openLuup concept of being parsimonious with resources… makes you realise just how profligate MiOS/Vera is, and shows that HA really shouldn’t need heavy-duty hardware, on the whole.

[Stuart]

So I created an additional variant of the openLuup image based on Alpine. And indeed, the image is a lot smaller. Total size including everything is 18MB now 8)

Wooo Hooo !!!

Brilliant - I will begin testing it tonight. If I spot anything I will report back.

[Stuart]

I must admit that I did not spend a lot of time on testing this variant, so please provide feedback.

Looking good !!
I have only found two things so far and both are relatively minor and easy to fix.

1. The folder /var/log/cmh is missing. As a result, log files are not being created.
2. Timezones are not implemented. Here is what I did:
   apt -U tzdata
   and then create a mount /etc/localtime :/etc/localtime
   This way it will automagically pick up the time from the host (assuming the host uses tzdata).

Note: it is possible to make this more general by passing in an environment variable e.g. TZ and then running some commands like:
ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone (not sure the second command affects anything btw)

In any case - for now I have gone with the first approach. So now my docker-compose.yml looks like this (remove the TZ variable and added the second mount):

version: '2'
services:
  openluup:
    container_name: openluup
    image: vwout/openluup
    restart:  unless-stopped 
    network_mode: host
    volumes:
      - <a convenient folder on your host>:/mnt/uploads
      - /etc/localtime:/etc/localtime

[vwout]

Good point about the timezones.

The logs mounts are equal to the Debian image. The openLuup logs are in /etc/cmh-ludl/logs/, or does it use /var/log/cmh (where Vera stores its logs) as well?

[Stuart]

or does it use /var/log/cmh (where Vera stores its logs) as well?

So it looks like (understand I’m new to this environment):

1. ALTUI looks to, by default, log to /var/log/cmh BUT, if running on openluup, then that location is on openluup
2. openluup logs by default to /etc/cmh-ludl/logs – so that’s where the logs for plugins installed on openluup go
3. Vera of course logs to /var/log/cmh

There may be some startup mapping for openluup that can be tweaked (not got there yet) …

If I go to ALTUI --> Misc --> osCommand --> Tail Logs (with your default install) it does not return anything. Adding that directory and restarting then provides log information.
The default command is: tail -n 50 /var/log/cmh/LuaUPnP.log
By the looks of it, it’s high level stuff, but “something” is definitely placing log entries there.

Executing tail -n 50 /etc/cmh-ludl/logs/LuaUPnP.log gives openluup information as well as locally installed plugins.

Not yet sure how to get to the vera logs from here :o

These may end up being some tweaks / mapping for the image as it gets figured out …

[Stuart]

So - we are on the right track, this from the official openluup documentation

openLuup also maintains another log, which is a subset of the main one, and contains only device variable changes, scene invocations, and workflow messages. Those entries are written to a log file, if the /var/cmh/ directory exists, in /var/cmh/LuaUPnP.log,

One other thing that would likely be helpful: set up /etc/cmh-ludl as a docker volume so that updates to the image do not over write that directory (there may be others …) or instructions on how to copy from that directory to a folder on the host and then mount the host directory to the image (I can do that tomorrow). I kinda prefer the later since it will survive an accidental docker reset.

[vwout]

Not yet sure how to get to the vera logs from here :o

Well, there is no Vera in scope of the openLuup container. openLuup could be linked to your Vera, but the Vera logs won’t be in the container on /var/log/cmh, but on your Vera. To access them from altUI running on openLuup that is on another box, you need to mount the folders from your vera. sshfs could be an option there.

[quote=“Stuart, post:16, topic:199649”]So - we are on the right track, this from the official openluup documentation

openLuup also maintains another log, which is a subset of the main one, and contains only device variable changes, scene invocations, and workflow messages. Those entries are written to a log file, if the /var/cmh/ directory exists, in /var/cmh/LuaUPnP.log,

[/quote] I can add the volume /var/cmh to have the Vera-like openLuup logs. Since you already have the full log, this may not add much other than the fact the removing information makes the remain stand out more, which could be usefull.

One other thing that would likely be helpful: set up /etc/cmh-ludl as a docker volume so that updates to the image do not over write that directory (there may be others …) or instructions on how to copy from that directory to a folder on the host and then mount the host directory to the image (I can do that tomorrow). I kinda prefer the later since it will survive an accidental docker reset.

This is something I discussed with akbooer in the prelude to publishing the image, see https://github.com/akbooer/openLuup/pull/13. Detaching openLuup from the image could reduce the image to basically a lua image with some plugins. The directory /var/cmh-ludl is a volume already. It means that docker creates the volume when the image is started first time, Docker will copy the contents of the folder to the newly created volume in that case. Using a pre-created (e.g. named) volume won’t work, since it will remain empty.

[akbooer]

FWIW, I generally add a sym link from /var/log/cmh to /etc/cmh-ludl/logs, and then the AltUI tail logs command works as is.

[vwout]

The images at vwout/openluup on Docker Hub are updated.

I applied the following changes:

• Providing /var/log/cmh Vera-style log folder (symlinked to openLuup logs)
• Debian based image: New tag (next to latest): ‘slim’
• Alpine based image: Enable timezone configuration via TZ environment variable
• Update to compose-file

[Stuart]

Looking very good !

I was getting errors from the Alpine version, basically complaining about syntax errors. I did not check the debian version.

I believe the error is in the file openLuup_reload_for_docker Specifically the line

function openLuupShutdown {

should be

openLuupShutdown(){

at least when I make that change, all seems well :o