Feature Proposal: Simplify installation by creating a collection of docker containers for common configurations.

Motivation

This doesn't change foswiki itself, but adds a cross-platform installation packaging that should simplify installation and configuration.

The question was asked on IRC recently - "Why is Foswiki so difficult to install", and following up with comments that at least we could create a docker container to automate installation.

Description and Documentation

I'm uncertain how to organize the container structure, but I think foswiki would be derived from the library Perl container, and link to supporting containers for web, database, etc.

• ubuntu
  • Foswiki-ubuntu
• Web (Apache, Nginx ...)
• Database (MySQL, Posgres, ...)
• SolrSearch engine

I'm not sure if it would be better to create specific containers for Foswiki-apache, Foswiki-nginx, etc, or to have just the foswiki container with -options to choose the web server, etc. We could also consider bundling extensions in alternate packages, for example Foswiki-NatSkin, which installs and configures Foswiki and the NatSkin ecosystem of extensions. With the CLI tools/configure, it would be relatively easy to fully configure extensions.

For the site that desires the latest perl, we might also make available foswiki-perl which uses the perl base, and installs all the deps using cpanm.

SvenDowideit has some initial docker containers as examples, but they install the web server directly as well. (https://github.com/SvenDowideit/dockerfiles/tree/master/foswiki)

We would also create a "foswiki" account on docker for distribution of the containers, and a "dockerfiles" repo under the foswiki account on github.

Examples

Impact

No direct impact on Foswiki internally, but could provide a much simpler installation process

This solves the Perl and CPAN version and availability issues. Basing Foswiki on Perl:latest would provide the latest version of perl, currently 5.24.0, And CPAN modules can be installed by apt, or cpanm as part of the container.

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

-- Contributors: GeorgeClark - 29 Oct 2016

Discussion

I don't like Docker. It feels totally alien. For most deployments it adds yet another virtualization layer on an already existing one. Most deployments are on a virtual environment already: vmware, virtualbox, .... Not Docker. So adding Docker inside a virtual image adds yet another layer of complication I don't want to handle. Therefore I'd prefer to build virtual images directly.

-- MichaelDaum - 30 Oct 2016

I've really just started looking at it, but it sure seems much more lightweight than a full on virtual environment like vmware, virtualbox, qemu. From what I can see it much more closely resembles a "jail" like we used to run on the f.o FreeBSD server. It does not virtualize the hardware, like on a classic VM, but uses the running OS kernel and layers on read-only and "copy-on-write" "union" file systems, such as aufs. With this layering of containers it's possible to delegate maintenance of various layers.

For example, with the current VMWare image, when I release a new foswiki, I usually have to also update the OS and other related packages, update any CPAN, clear the log and sanitize the install, reclaim freespace and re-compress the image. And users have to then re-install a completely new vm image, or do the updates themselves.

The Docker containers layer separate file systems. I'm still trying to understand it all, but it appears that base and intermediate containers can be readily updated "underneath" the top layer (which would be Foswiki).

-- GeorgeClark - 31 Oct 2016

Neither a containerized or virtualized Foswiki solves all our 'one-click' installation needs - not everyone could even use Docker or a VM. Mainly because of local customization (e.g. Nginx or Apache to name but one choice) and upgrade-install issues.

I'm going to discuss this further on the related DependenciesFreedom topic.

-- JulianLevens - 01 Nov 2016

I'm certainly not suggesting it's a one-size-fits-all, but it's probably a fairly low effort way to handle a subset of foswiki installations. Also it's a lot easier to automate this than the VM, which takes a considerable effort to refresh.

More than anything else, I think our biggest challenge is that we intermix code, configuration and data all along the same paths. And I've yet to hear of any realistic solution to our layout. Having the web server serve "pub" makes it even more difficult. And Extensions installation complicates things even further. It would be nice if our distribution files could be a shadow behind the local site's installed files, and let store merge them ... like a "union filesystem". Then System could truly be "read only".

-- GeorgeClark - 01 Nov 2016

Also need consider: the Docker solves only the "Linux" problem. Using the Docker on the FreeBSD is "strange", because the natural solution is using jails. For the macOS (OS X) for using the Docker need install the xhive (the mac port of bhyve virtualisation support from the *BSD project) - so unnatural also.

The ideal solution (from my point of view) is (as Michael wrote in the DependenciesFreedom).

$ cpanm Foswiki
$ genwiki new /some/path/to/the/wiki/tree      # this step is needed for easy generation of multiple wikis
$ plackup /some/path/to/the/wiki/tree/bin/app.psgi
#or
$ /some/path/to/the/wiki/tree/bin/run.pl

The run.pl is simple - for example as in the QuoVadisPsgiFoswiki

use Foswiki::Config qw($config);
use Plack::Runner;
my $runner = Plack::Runner->new;
$runner->parse_options( $config->get_server_args() );
$runner->run; # <-- this starts the plackup with the wanted "server args" from the config
# for example as:  plackup -s Starman -p 8080 .... etc.

The approach also solves the "chicken-egg" problem - we currently using a logic:

1. The user MUST specify exact paths for the ApacheConfigGenerator (of course)
2. and later, the boostrap guessing(!!!) this already exactly defined paths... (isn't strange?)

By me would be better:

1. specify the paths in some config file, which location is relative fixed to the run.pl
2. the use Foswiki::Config qw($conv) simply reads the config (our LSC)
3. and using the standardized Plack::Runner runs the server & the Foswiki
4. the chicken-egg problem solved

-- JozefMojzis - 01 Nov 2016

I want to drag this back to on-point. This is NOT a proposal to boil the ocean. It is a very simple narrow proposal to create some Docker containers that we can make available along side the existing zip, tar and VM packages. We would get an account in the docker repository and release them as officially supported containers.

So that's the proposal.

• Explore creating some docker containers. Based upon experiences there:
• Add the docker container build/release process to the build scripts / documentation.
• Make the containers available for sites that want to use docker. As another alternative. Not as a replacement to any current packages.
• And a commitment that we won't end up with orphaned containers as the project owns them, not an individual developer.
• This does not change foswiki core. The only additional work is on the Release Manager (or future release managers).

I added this as a proposal rather than just "doing it", so that we would not end up with the mess that is ArchivedOtherFoswikiInstallers

And I guess Michael has blocked this. Fine. On to a community vote. But I'll hold off on that until I actually have a bit more experience in how it all works.

-- GeorgeClark - 03 Nov 2016

Removed my concerns. Go ahead and create Docker images if you like. The CPAN discussion goes on at DependenciesFreedom.

-- MichaelDaum - 03 Nov 2016

Adjusting the install hierarchy. Choice of base image 'perl' is probably not the best choice for a casual installer. As pointed out elsewhere, it does a full compile of a local perl. See https://www.brianchristner.io/docker-image-base-os-size-comparison/ for a review of the common bases. As ubuntu has the highest usage, that's probably a reasonable one to use. It does have native perl, so there is no requirement to compile perl. While debian is the recommended base image (see https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/), it uses Perl 5.20. Ubuntu is not much bigger and has perl 5.22. For the remaining base images: centos, busybox, and alpine: perl not included. opensuse: perl 5.18

-- Main.GeorgeClark - 04 Nov 2016 - 17:21

Changing this to a Parked proposal.

-- GeorgeClark - 17 Jan 2018

This is really sad. Would install Foswiki in a heartbeat, really don't get how you can actually justify ignoring docker here (also think helm & kubernetes, for example).

-- AxelBock - 11 Nov 2018

ParkedProposal basically means that this FeatureRequest needs a CommittedDeveloper that is willing to contribute this feature. It does not mean that this feature is considered not worthwhile. It may mean that current developers indeed "ignore" docker in so far as there are other personal priorities that shift this feature down the list. So, please, if you like to see a docker installer, go ahead. You are more than welcome to do so. Feel free to contact us either here or on irc of slack in case you need assistance of any sorts.

-- MichaelDaum - 12 Nov 2018

I'm the developer who initiated the proposal, and then decided to park the proposal. There were a couple of reasons.

• The biggest technical challenge is that Foswiki isn't structured well for things like this: Docker, OS packages (dpkg, rpm), etc. It's discussed somewhat above, but mainly, Foswiki code is intermixed in with configuration and user data. So there is no easy clean way to switch in a new release. There have been small efforts along the way to tease things apart, but a lot more is needed.
• And on a personal note, I needed to step away from the project for a while. Nothing at all negative on Foswiki's side. I just had to pursue some other interests for a while. I'll be back at some point.

As far as building a docker install as is, it should be pretty straight forward. The hard part would be when you want to upgrade to a new version of Foswiki. If you search, you'll find several unofficial docker foswiki images out there.

And as Michael says, developers are welcomed and encouraged to contribute.

-- GeorgeClark - 13 Nov 2018

I concur with MichaelDaum and his earlier concern about yet another virtualization. That being said I have put together something at https://github.com/timlegge/docker-foswiki because well quite frankly RedHat sucks and since work is fairly standardized on it and it breaks some perl modules beyond repair, here I am.

The current image published on dockerhub has everyyhoing need to run the mPlugins I want like SolrPlugin, NatSkin, etc. and I have instructions there for deploying a Solr 5 server on the same docker host that can be up and running and testable in a few minutes. My plan is to move it towards production as soon as I get it ready.

-- TimothyLegge - 20 Nov 2018

I only chose Solr5 because of the cores and configsets of the current plugin seems to be targeted there and I was not enthused about figuring out those changes

-- TimothyLegge - 20 Nov 2018

Very nice!

-- MichaelDaum - 21 Nov 2018

I spent a little more time on the Docker image, It adds some extra perl modules and automatically saves a config and installs the following Plugins:

ActionTrackerPlugin ClassificationPlugin DiffPlugin DocumentViewerPlugin FilterPlugin GraphvizPlugin GridLayoutPlugin ImageGalleryPlugin ImagePlugin JQDataTablesPlugin JQMomentContrib JQPhotoSwipeContrib JQSelect2Contrib JQSerialPagerContrib JQTwistyContrib JSTreeContrib LdapNgPlugin LikePlugin ListyPlugin MediaElementPlugin MetaCommentPlugin MetaDataPlugin MimeIconPlugin MoreFormfieldsPlugin MultiLingualPlugin NatSkin RedDotPlugin RenderPlugin SolrPlugin TagCloudPlugin TopicInteractionPlugin WorkflowPlugin

https://github.com/timlegge/docker-foswiki

or run directly

sudo docker run --name docker-foswiki1 -d -p 80:80 timlegge/docker-foswiki

-- TimothyLegge - 13 Feb 2019

I have spent quite a bit of time working on creating and getting alpinelinux modules accepted into the alpinelinux aports. At this point I am waiting on three more to be accepted and a sizable percentage of the foswiki required modules are available as packages on alpinelinux:edge (numerous are in the testing repository).

I have updated the Dockerfile that I have at: https://github.com/timlegge/docker-foswiki/blob/saml-support-edge/Dockerfile. Once the last few modules are accepted I will do a build and push it to DockerHub

I am still waiting on perlmagick to be accepted as a package and I need to finish Net::SAML2 but most Foswiki installations can be full hosted on alpinelinux (or a alpine linux docker image) with native packages

-- TimothyLegge - 25 Apr 2019

A fully functional Docker version is available at https://github.com/timlegge/docker-foswiki and https://cloud.docker.com/u/timlegge/repository/docker/timlegge/docker-foswiki.

The only dependancy included that does not currently have a Alpine Linux package is Net-SAML2 (perl-net-saml2)

-- TimothyLegge - 8 Jul 2019

Hi All,

I would like to resurrect this proposal. I have read this and the discussion at DependenciesFreedom.

I think the discussion in this topic underestimates the potential of docker. I have used it a few times for demos and it is very effective, Once the image is built, you can run the image on any platform with a docker daemon . I think, but this is where I am without experience, that compose would be able to help mapping file systems, etc.

Anyway, this proposal is about providing a docker container. I see that in the first instance as a way to demonstrate Foswiki with a one click install. issues like upgrade and permanent storage will need to be worked out.

I have built a prototype to demonstrate what I have in mind in the DockerContrib at https://github.com/BramVan-Oosterhout/DockerContrib. A difference with the discussion above is that I suggest Foswiki does not store the Dockerfile or other supporting material. DockerContrib is a reference to docker images defined on the internet. Maybe I will rename it to DockerInstallContrib.

A user can clone DockerContrib and then can:

cd tool
sudo perl docker-install timlegge -build

And be presented with a link to a working Foswiki. (Tim's image)

• sudo perl docker-install dieken -build presents a link to an image in autobootstrap, and
• sudo perl docker-install bram -build presents a link to an application implemented with the wiki workbench.

All of these are demonstrations. and the docker-install script itself is hacked together and needs polish. But at this stage i just want to start the discussion again. Is anyone interested?

-- BramVanOosterhout - 20 Jan 2023

Commenting is disabled while not logged in