Using git to manage a Foswiki installation

This article is all about how you can use git to manage a Foswiki installation on your own server. You are assumed to know what git is, and what it's for.

For information about using git for Foswiki development, see HowToUseGit

Introduction

git is an excellent tool that can be used to help you manage your Foswiki installations. Some of the things it can do for you are:

1. Simplify upgrades
2. Make it easy to manage multiple installs
3. Protect you from extension installations (and even upgrades!) that you later regret
4. Record any local changes (hacks) you may make

The basic idea is that you should have two installations of Foswiki; a "test" or "staging" install, and a "production" install. Whenever you make a change (e.g. to install a new extension) you make it in the test install first, and when you are sure it works, you copy that change into the production install.

First you set up the test installation. Download the official Foswiki release package (we'll assume we are installing Foswiki 1.1.1 here) and unpack it in the normal way. We'll assume you have unpacked it in a directory called /var/www/test. Now, before you do anything else create a git repository in the installation, and check it in.

$ cd /var/www/test/Foswiki-1.1.1
$ git init .
$ git add .
$ git commit

This is now your release baseline, and you should create a new, local branch for it.

$ git checkout -b Foswiki-1.1.1

Despite the checkout, this command actually creates a new branch.

If you already have a working installation of Foswiki 1.1.1, you can now copy that installation over the top of this checkout. You will still have the baseline there, but the checkout will now include all your customisations. Add any missing files to the repository (but do not add bin/LocalLib.cfg or lib/LocalSite.cfg), and check it in.

Alternatively

Configure your Foswiki install using configure, adding extensions such as skins or plugins. You can choose to git commit the changes as you go along, or you can simply do one big commit when your initial configuration is complete.

Now, you need to duplicate this installation to create the production install.

$ cd /var/www/wiki
$ git clone /var/www/test/Foswiki-1.1.1
$ git checkout --track -b Foswiki-1.1.1

Of course the configuration will be different - the file paths are different, after all - but it's essentially the same installation (there's a trick for sharing most of the configuration, too, described below). User webs will be created in the production installation that won't immediately be available in the test installation. You can solve this by "soft linking" the directories (pub and data) containing the user webs into the test install, so you can see them while testing.

From now on, whenever you make a change to your installation, you make the change in the test install first, and check it in. You then run sanity checks to ensure that nothing is broken. At a quiet time, when all your users are asleep or playing Farmville, you restrict access to the production wiki, and refresh the production install so it picks up the changes committed to the git repository in the test install:

$ cd /var/www/wiki/Foswiki-1.1.1
$ git pull

You can have the test and production installs on different machines by setting the test server up as a git server. This simply involves giving ssh access to the machine - see http://tumblr.intranation.com/post/766290565/how-set-up-your-own-private-git-server-linux for (linux) instructions on how to do this.

Trick for sharing more of the configuration

This describes how to share common files between two installs; it does not describe how to run multiple Foswikis off a single install! For that, see VirtualHostingContrib

The above process doesn't let you share a key configuration file, lib/LocalSite.cfg. That's because this file contain absolute file paths, which differ between installs. However there's a trick you can use to make this file shareable, at the cost of some functionality in configure. The other key configuration file, LocalLib.cfg is usually very small, and there are some good reasons not to share it (for example, you will often want debugging enabled in the test server, but disabled in production) and we don't recommend trying to share it. So it's a good place to put installation-specific paths.

First, let's assume that our server is at "http://my.server", and you have configured the test install so it appears under "http://my.server/testwiki", and it points to the installation undef /var/www/test/Foswiki-1.1. Let's also assume the production installation is under "http://my.server/wiki", and it points to the installation (git clone) in /var/www/wiki/Foswiki-1.1. Thus you will refer to a page on the test wiki using a url such as http://my.server/testwiki/bin/view/Myweb/MyTopic and on the production server as http://my.server/wiki/bin/view/Myweb/MyTopic.

First, manually edit LocalSite.cfg on the "test" install. Find all the settings that contain absolute paths - for example,

$Foswiki::cfg{ScriptUrlPath} = '/testwiki/bin';
$Foswiki::cfg{ScriptDir} = '/var/www/test/Foswiki-1.1/bin';
$Foswiki::cfg{PubUrlPath} = '/testwiki/pub';
$Foswiki::cfg{PubDir} = '/var/www/test/Foswiki-1.1/pub';
$Foswiki::cfg{DataDir} = '/var/www/test/Foswiki-1.1/data';
$Foswiki::cfg{ToolsDir} = '/var/www/test/Foswiki-1.1/tools';
$Foswiki::cfg{TemplateDir} = '/var/www/test/Foswiki-1.1/templates';
$Foswiki::cfg{LocalesDir} = '/var/www/test/Foswiki-1.1/locale';
$Foswiki::cfg{WorkingDir} = '/var/www/test/Foswiki-1.1/working';

Replace all the common file path sections, /var/www/test/Foswiki-1.1 with a new configuration variable, $Foswiki::cfg{InstallPath}. Also replace all occurrences of the url base path /testwiki with $Foswiki::cfg{UrlPath}

$Foswiki::cfg{ScriptUrlPath} = '$Foswiki::cfg{UrlPath}/bin';
$Foswiki::cfg{ScriptDir} = '$Foswiki::cfg{InstallPath}/bin';
$Foswiki::cfg{PubUrlPath} = '$Foswiki::cfg{UrlPath}/pub';
$Foswiki::cfg{PubDir} = '$Foswiki::cfg{InstallPath}/pub';
$Foswiki::cfg{DataDir} = '$Foswiki::cfg{InstallPath}/data';
$Foswiki::cfg{ToolsDir} = '$Foswiki::cfg{InstallPath}/tools';
$Foswiki::cfg{TemplateDir} = '$Foswiki::cfg{InstallPath}/templates';
$Foswiki::cfg{LocalesDir} = '$Foswiki::cfg{InstallPath}/locale';
$Foswiki::cfg{WorkingDir} = '$Foswiki::cfg{InstallPath}/working';

Now add the following two lines to the top of LocalSite.cfg:

$Foswiki::cfg{InstallPath} = $::FOSWIKIINSTALLPATH;
$Foswiki::cfg{UrlPath} = $::FOSWIKIURLPATH;

This tells Foswiki to use values of the install path and the url path from LocalLib.cfg.

Finally add the following lines to LocalLib.cfg (anywhere, as long as it's before any final 1;)

$::FOSWIKIINSTALLPATH = '/var/www/test/Foswiki-1.1';
$::FOSWIKIURLPATH = '/testwiki';

Repeat this step for the configuration of the production install, modifying the paths as appropriate.

You can now safely add LocalSite.cfg to git, and share it between the installs, though you will need to be careful not to add any absolute paths in the configuration in future!

The downside of this is that configure will complain about the paths in "General Path Settings", where it will claim to be unable to find them.

How to handle upgrades

WORK IN PROGRESS - CAVEAT LECTOR

• Best way to handle upgrades?
• Do them on the same branch / create a new branch?
• Babar suggests removing the initial checkin, and checking in the new release
  • has the advantage of purging the old files
• Maybe we need to rethink the sharing approach, and put the base release and local changes on different branches. That we we could merge/unmerge them as required.

Discussion

There are a couple of tasks reporting errors with this approach. The issue is that the extensions installer, when run from the CLI, does not use an expanded configuration to find the directories. It's a bit of a chicken/egg issue because some sites use relocated bin and lib directories, such as cgi-bin, and /usr/lib/foswiki... Something needs to be anchored to a specific location or configure will be unable to locate itself.

• Tasks.Item10240 - extender.pl fails if $Foswiki::cfg{ScriptDir} contains $Foswiki::cfg variables
• Tasks.Item10249 - configure can't handle path names containing $Foswiki::cfg - variables