VirtualHostingContrib

Adds virtual hosting support for Foswiki.

Overview

VirtualHostingContrib provides the ability of using a single Foswiki installation to serve differents sets of webs under different host names. Such different hostnames by which users access different content are known as virtual hosts.

For users, it will look like they have an exclusive Foswiki installation. The system administrator, however, needs to maintain and upgrade a single Foswiki installation. This solution allows for instance service providers to offer Foswiki systems without having a linear increase in maintainance effort.

Installation Instructions

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server.

Open configure, and open the "Extensions" section. "Extensions Operation and Maintenance" Tab -> "Install, Update or Remove extensions" Tab. Click the "Search for Extensions" button. Enter part of the extension name or description and press search. Select the desired extension(s) and click install. If an extension is already installed, it will not show up in the search results.

You can also install from the shell by running the extension installer as the web server user: (Be sure to run as the webserver user, not as root!)

cd /path/to/foswiki
perl tools/extension_installer <NameOfExtension> install

If you have any problems, or if the extension isn't available in configure, then you can still install manually from the command-line. See https://foswiki.org/Support/ManuallyInstallingExtensions for more help.

Configuration

VirtualHostingContrib works by providing an alternative entry point to the classic Foswiki scripts interface. To use it, you need to configure your webserver so that requests to e.g. /foswiki/bin/view are actually handled by the /foswiki/bin/virtualhosts CGI script. Such script will play the role of all the other scripts, but before serving each request it will setup the appropriate environment for the virtual host being used.

Lighttpd

TODO

Apache

TODO

Using FastCGI

The virtualhosts script is a regular CGI script, and as such it may exhibit poor performance in sites with low computing power or high usage, just like regular Foswiki. VirtualHostingContrib provides a FastCGI version of the script.

To use it:

1. Make sure you have FastCGIEngineContrib installed and properly working on your system.
2. In your web server configuration for the virtual sites, replace the references to the foswiki.fcgi script provided by FastCGIEngineContrib with virtualhosts.fcgi provided by VirtualHostingContrib.
   • You will still need a conventional configuration for the base site for access to conigure.
3. Reload your web server configuration to apply the changes.

Virtual hosts management

This section describes how to perform common maintenance activities on your virtual hosts.

Settings

The following settings are available for this contrib in the Foswiki configuration interface:

Setting	Meaning	Default value
VirtualHostsDir	The directory in which you virtual hosts are stored	(empty) uses /path/to/foswiki/virtualhosts
ServerAlias	a mapping of aliases to real hostnames
DisabledServers	a list of servers that are disabled temporarily even though there's a valid environment present in VirtualHostsDir

The structure of a virtualhost

Each virtual host consists of a subdirectory of $Foswiki::cfg{VirtualHostingContrib}{VirtualHostsDir}, named after the hostname it is intended to serve. Take the following example:

  virtualhosts/
    example.com/
      ...
    mydomain.net/
      ...

In this case, there are two virtual hosts, one for the example.com domain and other for the mydomain.net domain.

Inside each virtual host directory, there are some directories that resemble the ones in the main Foswiki installation:

Directory	Stores
data/	topic data
pub/	attachments and other files that must be directly accessible by clients
templates/	custom templates for this virtual host only
working/	all sorts of temporary data

Virtual host specific settings

In each virtual host root, the server administrator can put a file called VirtualHost.cfg, which may contain Foswiki configuration variables. This way you can have different settings for different virtual hosts. For example, you can have some virtual hosts using the standard .htpasswd Foswiki user management, and others using LDAP to authenticate users.

To create your local configuration, just create a file called VirtualHost.cfg inside the virtual host directory. Inside the file, you use the $VirtualHost hash the same way you would use $Foswiki::cfg in the global Foswiki configuration.

Example:

# disable FooPlugin in this virtual host
$VirtualHost{Plugins}{FooPlugin}{Enabled} = 0;

# use LDAP contrib in this virtual host
$VirtualHost{PasswordManager} = 'Foswiki::Users::LdapPasswdUser';
$VirtualHost{UserMappingManager} = 'Foswiki::Users::LdapUserMapping';
# ...

Note that all occurrances of $Foswiki::cfg in the virtual host configuration will be replaced by $VirtualHost. This way you can still copy and paste configuration examples into your virtual host configuration, but the virtual host configuration won't affect the global configuration. A side effect of this is that you cannot do smart things such as:

$Foswiki::cfg{AuthScripts} .= ',view';

Warning: just like Foswiki's global configuration file LocalSite.cfg, virtual host configuration files are executed as Perl code. So the system administrator must be completely sure of what to put in that configuration file.

Creating/disabling/removing/renaming virtual hosts

To create a virtual host, a configuration wizard is now available under the Extensions -> VirtualHostingContrib -> {VirtualHostingContrib}{VirtualHostsDir} key.

Click the button, and enter the name of the desired hostname into the next dialog, and select the desired options. (Don't enter the hostname into the directory field.) Other fields in the dialog include:

Host template name

If you have one or more special virtual hosts named with a leading underscore (ie. _template), you can choose it in a drop-down box in the dialog. It will be copied over to the new virtual host.

If there is no _template virtual host or you chose the -none- option, the script will create the virtual host by copying files from the main Foswiki data; this is suggested if your installation contains only virtual hosts, so the main Foswiki data will always be clean. If the host supports symbolic links, the System, and the _empty and _default template webs will be linked rather than copied.

You can also create a _template virtual host manually by copying the files from a Foswiki release tarball, or by manually creating the data you want for every newly created virtual hosts.

If your installation is brand new and you plan to use the main Foswiki data afterwards, you can also create the _template virtual host using the script, and that will freeze a copy of the main Foswiki data for new virtual hosts created later. (caution If you are running from a git checkout using pseudoinstall, any symbolic links in the webs will be preserved in the copy.)

Config template name

If you have a text file(s) named with a leading underscore (ie. _template.conf) in your virtual hosts directory, you can chose the template config name in the dialog.

When you create a virtual host for example.com you'll also get a example.com.conf file which is copied from the selected _template.conf file, All occurrences of %VIRTUALHOST% will be replaced by the virtual host name (example.com in this example). Note that VirtualHostingContrib does not care about the contents of the file, it just takes the content of _template.conf, replaces all occurrences of %VIRTUALHOST% by the virtual host name.

Caution: The config file must be an executable perl file following the rules of the LocalSite.cfg, and as described above in #HostSettings.

Create .htpasswd file

By default, a new .htpasswd file will be created in the virtual hosts's data directory. If you un-check the checkbox and bypass creation of .htpasswd, registration will be disabled in the new virtual host.

Running the wizard with tools/configure

The CreateVHost wizard can be run from the CLI as well. The arguments are:

Parameter	Description	default	Example
name	virtual hostname to create	(required)	-args name=foo.mysite.com
createpasswd	Create the .htpasswd file.	no password file created	-args createpasswd=1
template	Template host to copy	host will be copied from master webs	-args template=_templateweb
templatecfg	Template file to copy into configuration	no config created	-args templatecfg=_template.conf

Caution: Always run this wizard using the web server's userid

sudo -u www-data tools/configure -wizard CreateVHost -method create_vhost_2 -args name=foo.site.com -args createpasswd=1
sudo -u www-data tools/configure -wizard CreateVHost -method create_vhost_2 -args name=foo2.site.com -args template=_template templatecfg=_template.conf

Managing virtual hosts

To disable a virtual host, add it to the list of DisabledServers. This is a comma-separated list of those hosts that are temporarily inactive and won't be served anymore until you either remove it from the DisabledServers list again or remove the files of the virtual host permanently.

To remove a virtual host, just remove the its files. If you are removing the virtual host for example.com, and your virtual hosts are stored in /var/lib/foswiki/virtualhosts (see #Settings), then it is enough to remove the /var/lib/foswiki/virtualhosts/example.com directory, plus any webserver configuration you have for that virtual host.

To rename a virtual host, you just have to rename it's directory, and if it's the case, change the configuration files accordingly. Beware that after renaming a virtual host its data will no longer be available at the old domain name.

To create an alias name for a virtual host, add a name mapping to the ServeraAlias hash of your global configuration in LocalSite.cfg. For instance use the following to access the example.com domain using the a different network interface as well:

$Foswiki::cfg{VirtualHostingContrib}{ServerAlias} = {
  '127.0.0.1' => 'example.com',

Note that you also need to configure your http server to listen to the alias name to make use of this feature.

Warning: in most cases where both domains, the canonical domain and the alias one, are reachable on the same network interface it is preferable to use a http redirect from one domain to the other instead of creating another domain that basically serves the same content. Also, SSL certificates appear to be invalid when served from the wrong domain. Don't use server aliases for in that case.

Using a server alias does make sense when the same domain isn't reachable on a second network interface from where you'll only be able to reach the server using the alias name. This might also be the case when maintaining the site on a different server offline and syncing content later on to the machine that is able to serve the canonical domain.

Using the original shell script to create a Virtual Host

You can use the provided script at tools/virtualhosts-create.sh. To create a virtual host for the example.com domain, you should run it like that:

$ ./tools/virtualhosts-create.sh example.com

If you run this script as a user different from the one that runs the Foswiki code (e.g. you create the virtualhost as root but your web server runs as www-data or nobody), the data/ and pub/ directories will be their ownership properly set to the correct user (i.e. www-data or nobody).

Running command line tools against the virtual hosts

VirtualHostingContrib comes with a set of tools to run specific tasks on the commandline, such as

• virtualhosts-mailnotify: a vhost-aware replacement for the standard mailnotify script used in cronjobs to send out mail notifications
• virtualhost-statistics: a vhost-aware replacement for the standard statistics script used in cronjobs to update the WebStatistics files in each web
• virtualhosts-rest: call a rest handler for a specific virtual host on the commandline
• virtualhosts-tick-foswiki: a vhost-aware replacement for the standard tick-foswiki maintenance script used in cronjobs
• virtualhosts-view: render a page of a virtual host on the commandline
• virtualhosts-jsonrpc: a vhost-aware commandline tool for the JSON-RPC interface of JsonRpcContrib
• virtualhosts-convert-charset: Run the CharsetConverterContrib convert utility against virtual hosts when migrating From Foswiki 1.x to Foswiki 2.x.
• virtualhosts-convertTopicSettings: Utility for converting from Foswiki 1.x style deprecated DENY rules to Foswiki 2.x * Wildcard ACLs

These scripts are located in the tools/ subdirectory of your Foswiki installation and can be called from the commandline. They all understand the usual parameters that their non-vhost-aware counterparts have. In addition, they all take an optional host parameter that lets you specify on which virtual host to run the command. When no host parameter is specified, the command will be run on all known virtual hosts known.

For example, to run a REST handler check implemented by a MyNewPlugin extension, use

cd tools
./virtualhosts-rest host=example.com /MyNewPlugin/check topic=Main.WebHome option=foo

(TODO: virtualhosts-mailnotify doe not yet understand host).

How it works

This section describes the internals of VirtualHostingContrib, and is intended for developers and system administrators. If it's not your case, please feel free to skip it.

Tweaking the global configuration

In order to serve different sets of webs within the same Foswiki installation, we need to tweak the Foswiki configuration to change some settings, such as DataDir and PubDir (the directories where topic text and attachments are stored, respectively).

The Foswiki configuration is loaded during compilation time, but when using persistent execution models (e.g. ModPerl, FastCGI), the Foswiki Perl code is compiled only once for multiple requests. This wat, the needed configuration tweaking must be done once for each request, and moreover must not not leave any trail for the next request, which will possibly be handlind a different virtual host (and thus different settings for DataDir, PubDir, etc.).

We need something like this:

               +-----------------+
               |                 |
    +----------+ Foswiki startup +
    |          |                 |
    |          +-----------------+
    | Main
    | configuration
    | loaded
    |
    |           Configuration cleaned
    |    +---------------------------------+
    |    |                                 |
    |    V                                 |
+------------------+             +---------+---------+
| Incoming Request +             | Outgoing Response +
+--------+---------+             +-------------------+
         |                                 ^
         | Configuration ajusted           |
         | for current virtual host        |
         |                                 |
         |                                 |
         |     +--------------------+      |
         |     |                    |      |
         +---->| Request processing +------+
               | (conf. loaded)     |
               +--------------------+

Since all Foswiki-specific processing happens inside the "Request processing" box, then if we manage to tweak the configuration before it is run and to restore it after the request is processed, we are all set. For that, we hook in the Foswiki engines system.

The Foswiki engines system

All requests are processes by the current Foswiki engine in the following way ($self is the engine object):

1	A Request object is prepared,	$req = $self->prepare()
2	The processing is delegated to the Foswiki::UI module. This module detects what was the called script (view, edit, attach etc.), instantiates a Foswiki session and does all of the actual processing to produce a result for the client. In special, this phase is the only one in which the vast majority of the Foswiki configuration matters.	$res = Foswiki::UI::handleRequest($req)
3	The engine does a cleanup and sends the response back to the client	$self->finalize($req, $req)

So, our objective is to wrap step 2 as the figure above shows: we adjust the configuration for the current request just before entering Foswiki::UI::handleRequest, and restore the configuration just after it finishes.

The Perl local() construct

The Perl local() construct enables the programmer to set a global variable until the end of the current block. After the current block exits, the value is restored to its original value. This can be also used with hash elements: in this case, only the elements changed with local are restored when the scope ends.

Example:

use Foswiki;
# Assume that DataDir is set initially to "/path/to/foswiki/data"
print $Foswiki::cfg{DataDir}, "\n"
{
   local = $Foswiki::cfg{DataDir} = "/path/to/virtualhosts/example.com/data";
   print $Foswiki::cfg{DataDir}, "\n";
}
print $Foswiki::cfg{DataDir}, "\n"

The above example will print the following:

/path/to/foswiki/data
/path/to/virtualhosts/example.com/data
/path/to/foswiki/data

This way we can override the settings we need, and also gain the cleanup step for free since Perl will automatically revert all changes done with local when the block ends.

For more information on local(), please refer to "Temporary Values via local()" in the perlsub(1) manpage (you must have Perl's documentation installed in your system).

Wrapping it up

What VirtualHostingContrib does is to wrap the Foswiki::UI::handleRequest so that the configuration is tweaked just before actually handling the request, using the approach described above. The two extra CGI scripts provided are equal to the ones in Foswiki core and FastCGIEngineContrib, but after loading Foswiki::UI, they load the VirtualHostingContrib module that does the wrapping around Foswiki::UI::handleRequest.

Info for developers

This section presents tips for developing of virtualhost-friendly code:

reading Foswiki configuration values

When you need to read Foswiki configuration options, make sure you do not do it during compile Time. Since virtualhost requests temporarily redefine part of the configuration, if you freeze a configuration value during a request for virtualhost A, you'll probably have problems when using that value in a request for virtualhost B.

Example: if you do something like this:

use vars qw($RE);
BEGIN {
  $RE = qr/$Foswiki::cfg{SomeOption}/;
}

If this code runs under ModPerl, FastCGI or other persistent engine, Then the $RE variable will be initialized in the first time that code is loaded, and that may be in the context of a specific virtual host. In a future request, $RE will still contain a value derived from the value of $Foswiki::cfg{SomeOption} for the original virtualhost, even if the current virtualhost redefines $Foswiki::cfg{SomeOption}.

To fix this, you should avoid initializing values that depend on the configuration during compile-time. Instead, compose the values you need during runtime:

sub do_stuff {
  my $RE = qr/$Foswiki::cfg{SomeOption}/;
  // use $RE here ...
}

or in a OO context:

sub new {
  // ...
  $this->{RE} = qr/$Foswiki::cfg{SomeOption}/;
  // ...
}

sub method {
  my $this = shift;
  // use $this->{RE} ...
}

Avoid using the 'o' regular expression flag together with configuration values

If you need a regular expression that is composed from values in the configuration, make sure you don't use the 'o' flag, which causes the regular expression to be compiled only once. For example, in the following code the $Foswiki::cfg{SomeOption} will be expanded only in the first time the expression is evaluated:

my $RE = qr/$Foswiki::cfg{SomeOption}/o;

Even is a virtualhost redefined $Foswiki::cfg{SomeOption}, the value used in that regular expression will be the value defined in the context of the virtualhost that was being processed during the first time that expression was evaluated.

So, avoid the 'o' flag when interpolating configuration values in regular expressions. The following real code shows the problem:

use vars qw($GLOBAL);

$GLOBAL = 'Main';

sub f {
  my $re = qr/(?:$GLOBAL\.)/o;
  print $re, "\n";
}

f();
$GLOBAL = 'Users';
f();

The two calls to f() will print the same value, based on the value $GLOBAL has in the first time the regular expression was evaluated:

$ perl test.pl
(?-xism:(?:Main\.))
(?-xism:(?:Main\.))

If you remove the 'o' flag from the regular expresion, you will obtain the desired result:

$ perl test.pl
(?-xism:(?:Main\.))
(?-xism:(?:Users\.))

Info

Many thanks to Colivre for supporting this work.

Dependencies

Name	Version	Description
File::Copy::Recursive	>=0	Required.
Foswiki::Contrib::FastCGIEngineContrib	>=1.20	Required.

Change History

05 May 2022	rewrite to prevent monkey-patching the core engine; support warming up in newer FastCGIEngineContrib; fixed multiple memory and filehandler leaks; added virtualhosts-save tool
20 Jun 2017	Foswikitask:Item14424: remove pending registrations. Foswikitask:Item14423: adjust to newer versions of MailerContrib
30 May 2017	Foswikitask:Item9958: virtualhosts-create.sh has issues. Foswikitask:Item11629: Add config checker and fail more gracefully. Foswikitask:Item14408: VirtualHostingContrib needs to support convertTopicSettings. Foswikitask:Item14409: Add checker and wizard to help manage virtual hosts. Foswikitask:Item14410: VirtualHostingContrib not compatible with CharsetConverterContrib. Foswikitask:Item14411: virtualhost.fcgi out of sync with foswiki.fcgi.
18 Mar 2014:	exclude server aliases running a command on all hosts; added quiet commandline parameter
10 Jul 2013:	added feature to temporarily disable virtual hosts; -- Foswiki:Main/MichaelDaum
01 Jul 2013:	added support for WebStatistics and a couple of other commandline tools
04 Jul 2012:	added foswiki maintenance tools for virtual hosting; -- Foswiki:Main/MichaelDaum
20 Aug 2011:	added server aliases; fixed interfacing new logger api; added run_on() api; -- Foswiki:Main/MichaelDaum
02 Nov 2010(2):	code cleanup and performance optimization -- Foswiki:Main/AntonioTerceiro
02 Nov 2010:	added support for listening on non-standard ports; performance optimization -- Foswiki:Main/AntonioTerceiro
16 Jul 2010:	first release (Antonio Terceiro)