How To Write A .spec File

Note: This How-To is written for Foswiki 1.2 and later. For older versions of Foswiki, you are probably better off reading DevelopingPlugins

From the earliest days, Foswiki has held system configuration information in an executable perl file called lib/LocalSite.cfg. This file is simply a series of key-value settings. In the beginning the only way to create this file was to manually edit a template, and this approach is still possible today. However things have moved on a bit since then, with the development of the configure script, that provides a web interface to the configuration.

As the configuration of Foswiki became more complex we needed an abstract model of the configuration data, so we could use it to drive user interfaces, and generalise the checking process for different kinds of configuration item. Thus the .spec file was born. Foswiki uses a master .spec file called lib/Foswiki.spec, and each extension may have an optional Config.spec. The format of these files is identical.

DevelopingPlugins gives a high-level view of the syntax of .spec files, and most developers will not need to explore beyond the capabilities described there. This guide goes much more deeply into the nature and syntax of .spec files, and is mainly for the use of core developers.

Basic Concepts

The basic idea behind .spec files is that they are executable perl files that specify default values for configuration keys. For example,

$Foswiki::cfg{DataDir} = '/my/data/directory';

By using Perl syntax we leverage the ultra-fast Perl parser to best effect, and remain copy-paste compatibility with LocalSite.cfg.

Structured comments are used in the .spec files to provide additional information about individual keys, and to subdivide the configuration into sections that are manageable in a UI. Additional structured comments provide information about the keys, such as type.

There are two places where .spec files are used. Firstly, the master lib/Foswiki.spec is the schema for the core Foswiki configuration. It contains specifications for all the controls that affect core modules. Secondly, extensions can have their own .spec files, called Config.spec, which insert additional configuration items specific to the extension.

Naming of Config.spec elements

when building your extension, there are several choices on where to place the elements in the $Foswiki::cfg hash:

Location	Example	Notes
Under the {Extensions} namespace	$Foswiki::cfg{Extensions}{BathPlugin}{PlugType}	This is the recommended location, It is recommended for all new extensions, and strongly recommended for non-Plugin type extensions (Contribs, Skins, etc).
Under the {Plugins} namespace	$Foswiki::cfg{Plugins}{BathPlugin}{PlugType}	This is traditionally where foswiki organizes all plugin settings. Foswiki automatically populates two settings for Plugins (and only plugins): {Plugins}...{Module} {Plugins}...{Enable} which must not be defined in the Config.spec file. We no longer recommend using this namespace for custom settings.
Under the root namespace	$Foswiki::cfg{BathPlugin}	Not Recommended Historical extensions place settings under the root, it results in a very cluttered configuration.

Config namespace Rules going forward:

• Existing extensions should probably not be restructured, to avoid configuration migration issues.
• If developing a "Contrib" or "Skin", configuration should live under {Extensions}, even if it has a Plugin component.
• If developing a "Plugin", it can use {Plugins} space but this is no longer recommended.
• Never store settings at the root level.

Also, don't confuse the layout within Configure tabs with the hierarchy within the configuration hash. Settings are organized in the Configuration Tabs based upon the heading hierarchy of the Spec file, not the configuration hash structure. It is strongly recommended to stick to one location for settings for any extension. Try not to split settings into multiple areas of the configuration hash, except when the settings are "Extending" a core feature like the Store. All of these below settings will appear under the Extensions -> BathPlugin tab. (but this is certainly not recommended).

---+ Extensions
---++ BathPlugin
$Foswiki::cfg{Plugins}{BathPlugin}{PlugType} = 'rubber';
$Foswiki::cfg{Extensions}{BathPlugin}{ToyType} = 'duckie';
$Foswiki::cfg{foobar} = 'some archaic root-level setting';

Structure of a .spec file

As mentioned above, a .spec file is an executable Perl file. It consists of:

• Structured comments called Sections (lines beginning with #)
• Configuration key specifications and assignments
• And should end with a single line consisting of 1; to ensure that the file evaluates to "True" when executed.

Sections

A .spec file is structured into sections using TML syntax comments, such as

# ---+ Section Title
# This is a comment describing the section. The first paragraph is special, and will always be shown in interfaces (other paragraphs may be expanded using controls in the interface).
# The comment is formatted using a subset of the TML (Topic Markup Language) used in Foswiki topics. You can also embed raw HTML.

These comments are used to create SECTION objects in the model when it is loaded, and thus drive the construction of a UI, but otherwise have no impact.

Keys

The real meat of a .spec is in the specification of the keys. The key specification is placed in comment before the configuration key, and consists of a type declaration followed by a comment. At it's simplest, this is just a type name in stars, followed by a comment:

# **NUMBER**
# This describes the setting. The first paragraph is special, and will always be shown in interfaces (other paragraphs may be expanded using controls in the interface).
# The description is formatted using a subset of the TML (Topic Markup Language) used in Foswiki topics. You can also embed raw HTML.

The actual key looks like this:

$Foswiki::cfg{Extensions}{ConfigurePlugin}{ExampleKey} = 42;

You can also specify a key that has no default (will be undefined) by placing the key in a # comment

# $Foswiki::cfg{Extensions}{ConfigurePlugin}{ExampleKey} = 42;

DevelopingPlugins has more information on the types available and the basic options that can be used with them.

The simple NUMBER type specification shown above can be extended by a number of attributes. Most of these are described in DevelopingPlugins, but there are a few more not described there that are used in Foswiki.spec, and are worth saying a bit more about.

• FEEDBACK - allows you to specify additional feedback requirements for the key. You can have as many FEEDBACK attributes as you want, each specifying the attributes for an action to be associated with the key (usually a button). A FEEDBACK is simply a semicolon-separated list of key-value pairs. These pairs specify the attributes for a button:
  • label the text to be used for the button
  • method the checker / wizard method to be invoked by the button
  • wizard (optional) if not given, the method is called from the checker package for the item. If a wizard is given, then the wizard package is looked up in lib/Foswiki/Configure/Wizards and the method run from there.
  • title - an optional help string for the button.
• CHECK - is used to specify additional parameters to the checker for the key. This allows parameterisation of a generic checker without the need for a specific checker package for the key. A CHECK attribute specifies a space-separated list of key-value pairs which have meaning to the generic checker. Each pair consists of an attribute name and an optional value. The value is a comma-separated list of keywords, numbers or strings.
  All checkers support the 'also' option (see CHECK alsobelow). Different generic types have different CHECK options:
  • PATH - supports a comma separated list of permissions and filters in the form of
    perms:<permstring>,<'filter regex'>,<permstring>,<'filter'>...
    If perms is specified, at least one permstring is required. filter is optional, if not specified, all files and directories are checked.
    
    • permstring - validate file permissions using entered code. Code is a combination of
      • F(ile), D(irectory) Key is a valid file or directory name - only used in the first entry, used when field is checked directly.
      • The remaining permissions are validated in a recursive path search, FEEDBACK = "method='validate_permissions';title='Validate file permissions.'"
        • r(ead), w(rite), x(ecute), match file permissions
        • f(ile permissions match {Store}{filePermission},
        • d(irectory permissions match {Store}{dirPermission},
        • p(references topic exists in directory) e.g. rwdp
    • filter - regexp that filters files for which this check doesn't apply. Entered as a quoted string. Enter '*' as a place holder to match all files.
  • URLPATH
    • expand expand $Foswiki::cfg variables in value
    • nullok allow item to be empty
    • notrail disallow trailing /
    • user Permit user@host syntax
    • pass Permit user:pass@host syntax
  • URL - as URLPATH with the addition of:
    • schemes schemes allowed. Default: schemes:http,https
    • authtype authority types allowed in item (from host, ip, hostip). Default: authtype:host
    • parts - list of parts allowed in item (from scheme, authority, path, query, fragment). Default: parts:scheme,authority,path
    • partsreq - list of parts required in item
  • NUMBER
    • radix radix (2-36), specified in decimal. Default: radix:10
    • min minimum in specified radix
    • max maximum in specified radix
  • OCTAL
    • min minimum in base 8
    • max maximum in base 8
  • STRING
    • min minimum length, Default min:0
    • max maximum length
    • accept list of regexp, one of which must match
    • filter list of regexp, none of which may match
  • EMAILADDRESS
    • list if present, permit a list of comma-separated addresses
  • DATE
    • zone - default timezone (from utc, local)
    • raw - assume raw user input (don't normalize to ISO format
  • <Generic checks>
    • emptyok - The string type value of '' is permitted. (Default is noemptyok)
    • undefok - The value can be undefined. It will be omitted when written to the config if no value is assigned. (Default is noundefok)
• CHECKER - normally a configuration item is checked by a checker found by looking up the keys for the value and loading a package. This allows you to redirect to a different checker. It is useful when two or more keys are inter-dependent and share identical checking requirements.
• CHECK_ON_CHANGE - specifies a list of keys to check when the value of this key is changed (dependencies). See CHECK_ON_CHANGE below.
• SPELLCHECK - if given, and the browser supports it, then allow spell-checking on the value of the item.

CHECK_ON_CHANGE

All types support the CHECK_ON_CHANGE option, which allows you to specify a (comma-separated) list of other keys. This specifies that if the value of any of these other keys changes, then this key needs to be checked as well. For example,

**STRING CHECK_ON_CHANGE="{Y}"**
$Foswiki::Cfg{X} = 1;

tells the code that when the value of {Y} changes, then {X} needs to be checked as well.

You can also specify the reverse relation using the CHECK option. The also switch specifies a list of other keys that will need to be checked if the value of this key changes. For example,

**STRING CHECK="also:{X}"**
$Foswiki::Cfg{Y} = 1;

is the equivalent of the CHECK_ON_CHANGE example above.

ENHANCE

Definitions of type ENHANCE cause the documentation to be added to an existing configuration definition. For example, the following example is the base definition for the Store implementations. It is essentially undocumented in the Foswiki.spec file.

# **SELECTCLASS Foswiki::Store::* **
# Store implementation.
$Foswiki::cfg{Store}{Implementation} = 'Foswiki::Store::PlainFile';

However this definition can then be expanded from the definition in a Config.spec file, For example in the PlainFileStoreContrib/Config.spec file, the documentation is "Enhanced". The documentation in the Config.spec file is merged with the definition in Foswiki.spec.

# **ENHANCE {Store}{Implementation}**
#    * PlainFile - # (installed by the PlainFileStoreContrib) is just about the simplest store
...
#      is the reference implementation of a store. 

Pluggables

As well as the simple mechanism described above for including keys in the configuration, you can also use pluggables. These are code modules located in lib/Foswiki/Configure/Pluggables that are used to automatically construct blocks of keys by querying the environment. For example, the LANGUAGES pluggable is responsible for determining what languages are available on the system, and creating keys that correspond to each language. Pluggables are invoked in the .spec by quoting the name of the pluggable in single stars:

# *LANGUAGES*

Checkers

At this point we need to introduce checkers. A checker is a perl module which, given a Foswiki configuration, will check that the values of one or more keys are within acceptable constraints. This is like a validator in Javascript, but Checkers are designed to go a lot deeper than simple syntax checking. For example, a checker might determine whether a URI is accessible without authentication, or check that a number of inter-related keys describe a useable configuration. Checkers are perl modules, and are created one-per-key under the lib/Foswiki/Configure/Checkers directory, in a directory structure that matches the position of the key in the configuration data. Take a look at that directory, and it should be fairly obvious.

As well as the checker modules for specific keys, there are also a number of generic type checkers. For example, lib/Foswiki/Configure/Checkers/STRING.pm is a generic checker for STRING-type configuration items. These generic checkers can be parameterised using information from the .spec file, using the CHECK option. For example,

# **URL CHECK="parts:scheme,authority \
#              partsreq:scheme,authority \
#              schemes:http,https \
#              authtype:host" **

This is the type specification for a configuration key that will contain a URL. The CHECK clause specifies what the URL must contain in order to pass the check. Individual generic type checkers support different CHECK options, as described above.

Checkers normally declare a single function, check_current_value, that checks the current value of the item. Checkers never :

• Change the configuration, in any way (no writing files, no changing %Foswiki::cfg, nothing)
• Assume anything about the environment (e.g. they never print to STDOUT or STDERR, and don't assume CGI is available)
• Assume Foswiki is available. %Foswiki::cfg is there, but no other part of the Foswiki code can be assumed.
• Run for a long time - they must be fast!

Checkers may also declare additional wizard methods that are referenced via the FEEDBACK mechanism described next.

Wizards

Wizards are code modules that perform functions beyond simple checking. Wizards are expected to be able to modify the configuration, and may write files, change permissions etc. they may run for a long time e.g. checking permissions.

Wizards are specified through the FEEDBACK attribute. For example,

# **EMAILADDRESS FEEDBACK="label='Send Test Email';wizard='SendTestEmail'; method='send'"**

The FEEDBACK attribute is a semicolon-separated list of key-value pairs, where the value is a quoted string (" and ' both supported) or a number. The following keys are supported:

• label - specifies a label that identifies the wizard.
• title - a wordy description of what the wizard does.
• wizard - specifies a code module to load from lib/Foswiki/Configure/Wizards that contains the wizard method. Default is to look for the method in the checker.
• method - (required) specifies the name of the method (function) to call
• auth - if auth=1 then authentication is required (the wizard expects to be passed username and password, though what it does with them is up to the wizard)

As mentioned above, Checkers may also define wizard methods, which is useful when information about the specific key is relevant to the wizard. If the wizard attribute is not given, then it will look for a function called method in the checker code package.

The full specification of wizards are documented in the lib/Foswiki/Configure/Wizard.pm module. Wizards in lib/Foswiki/Configure/Wizards can report back to the caller and include information about what aspects of the configuration were changed (whcih keys have new values as a result of running the wizard).

User Interfaces

The .spec format is designed to specify the schema of a configuration without making any assumptions about the user interface being used to modify the configuration. In fact there are two interfaces available, a command-line interface ( tools/configure ) and a web-based interface implemented as a Foswiki extension (ConfigurePlugin). Other interfaces may become available in the future.

tools/configure

tools/configure is a perl script that provides a fully-featured command-line interface that lets you explore and modify the configuration, call wizards and install/remove extensions. It is internally documented - run

$ perl tools/configure -help

for full help information.

ConfigurePlugin

Prior to release 1.2 Foswiki shipped with a monolithic user interface that provided web access to the configuration. This UI has been replaced by the much lighter (and much more efficient) ConfigurePlugin.

The ConfigurePlugin consists of a server-side plugin module and a JQuery user interface. The server-side plugin provides a number of jsonrpc entry points that support the retrieval of configuration information, checking, the invoking of wizards, and the saving of updated configurations. As far as possible the server side code is kept clear of any user interface assumptions.

The client-side Javascript (JQuery) module is responsible for all the presentation of the configuration data. It uses the jsonrpc API of the plugin to perform queries over the configuration data.

-- CrawfordCurrie, GeorgeClark