Feature Proposal: Security checklists for the uninitiated

Motivation

Foswiki's out-of-the-box settings are inappropriate for public sites. Make it easier/more obvious to set up for this type of installation.

For a long time I've run a publicly accessible wiki. It's actually not hard to do (i.e. avoid becoming a spam farm), if you're vetting new users so that nobody has CHANGE permission anywhere by default, and your NewUserTemplate denies CHANGE or even VIEW from WikiGuest (until they're vetted into some TrustedGroup).

But there is a big difference between the out-of-the-box ACLs (allows WikiGuest CHANGE access) and {AuthScripts}. More and more these days, a public Foswiki will need to remove /bin/rest from {AuthScripts} so that the public can visit and do nifty AJAX-y things without registering and logging in.

That's exposing a massive misunderstanding I've had for two years now, about Foswiki and its default treatment of WikiGuest. If you do this (remove /bin/rest from {AuthScripts}), rest handlers (unless they specify authenticate => 1) may be exploited to perform updates for WikiGuest. This includes the new (trunk-only) AJAX enabled CommentPlugin.

Description and Documentation

Not sure how to do this yet: wiki-apps? Configure checker? Specify a configuration "profile" at install-time (first configure screen) of either "intranet" or "public"? Both want the best security, but a public site will want to limit anonymous/guest contributors.

I don't think documentation is enough, but SecurityChecklists aims to be the beginning of that documentation while we flesh it out.

• Eliminate {AuthScripts} setting entirely. Requires a separate DeprecateAuthScriptsSetting proposal. If we really want to keep it, it should be {UnAuthScripts} instead (see AllowGuestsToUseRESTAsDefault). We might also DeprecateBinScripts too - this has already been done if you're using FastCGIEngineContrib (only the /bin/foswiki.fcgi script is ever used).
• Remove rest from {AuthScripts}, so that WikiGuest can use nifty AJAX stuff. Configure checker.
• Ensure there's no WebPreference-level CHANGE permission implied for WikiGuest anywhere. We can't trigger things from configure currently, so this has to be a wiki-app. This will flag problems on an out-of-the-box wiki.
• If configuration profile is public but only allowing trusted contributions, and registrations are enabled, check that NewUserTemplate hides the user topic from WikiGuest (bots/crawlers). Could be a configure checker or wiki-app.
  • Also warn if RegistrationAgent is unable to create new user topics in Main web

Examples

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

-- Contributors: PaulHarvey - 11 Jun 2011

Discussion

Agree. There are other issues. Things are also not appropriate for many intranet sites. See Tasks.Item10873, where I raised the following additional issues:

• lib/Foswiki/Configure/Checkers/WorkingDir.pm has hard-coded 0755 permissions. This needs to be configurable.

• Configure and installers apply other policies assuming an open system - world read for directories and files, and also assumptions about group permissions. These don't fit enterprise (or my personal) environments and need to be configurable.

• Configure rejects group 's' permission on directories - which is very useful to get created files to inherit the site's permissions policy; e.g. files created in working must not have world access, regardless of the webserver's umask.

• Selinux needs first class support in install (including plugin/extension) and configure. Existing topics/scripts make every change a fight with manual overrides.

• One non-obvious selinux divot is pseudo-install, which creates absolute links. It should create relative links. You get in selinux trouble where you have your svn checkout under your home directory (normally blocked by selinux from webserver), but core/ softlinked from /var/www and labeled for webserver access. With a relative link, the webserver walks up to your (svn) core root, then down to the related directory. No complaints. But with absolute links, it tries to walk from root down thru your home directories. Selinux alarms and blocks access.

• Speaking of which, relative links really should be everywhere in configuration - besides selinux, it also makes moving an environment from production to development for debugging a lot easier.

• I posted some (really ugly) scripts for selinux support over in Tasks.Item10873.

(I don't know if you want to include all of 10873 here, or just point to it. There is more detail.)

I'm not arguing that "open" is bad and "closed" is good - but in "closed" environments, the installers and configuration shouldn't be fighting admins every step of the way. And selinux is the out-of-the-box default for fedora (and I belive RHEL). Even in an open environment, it helps contain malware.

I also think that rather than checklists (or perhaps I mean, to minimize the checklists work and intimidation factor), we can automate a lot of the policy choices. And it's not that hard. As a starting point/proof of concept:

In my work on the task framework, I have had to create a post-installer. Some of its ideas may help here - I'll be providing that code when I have a kit ready. But the basic ideas are:

MANIFEST is extended with some parsable comments. These provide:

• Selinux labels for following files

  • Normally labels are inherited from the directory, but xWiki doesn't organize files into directories that map perfectly to the right labels. So some overrides are necessary.

• User.Group overrides for following files

The installer auto-detects whether selinux is active and won't try to label if it's not. And it has the usual command line overrides.

Also, it has the concept of an installation umask. Here, the idea is that the MANIFEST lists the permissive (open system) permissions as usual. But the installer has a umask for files that it will apply. So foo.pm is listed as 0644 in MANIFEST as usual, but the installer will &~ that with the files umask. E.g. if your installation policy is open, you can make your files umask 002, and thinks will work as today. If your policy is secure, make your files umask 007, and the permissions in the manifest are modified to no world access. There's a second umask for directories, so one can decide on whether they have world read, world execute, or none. And actually, I did this as "permissive" or "not permissive" - it's a switch, not an octal value for now.

It would be straightforward to put these policies into configure for the normal installers. To start with, 'user.group' for file ownership, 'files umask' and 'directories umask'. Selinux type labels (which may vary by OS/release) for the classes of files we use.

Then in the initial install, after configure (which could run permissive), we run a script like my post-installer that fixes things up to match the configuration. Thereafter, installs, including extensions, obey the stored config. And if policy changes (via configure), the post-installer is re-run to make the disk agree with the new policy.

The following is a work in progress, but may be helpful. (One reason things are a bit slower than usual is that since I'm starting with a clean environment, I have a policy that I will make no change unless it's in an automated script. So every chmod, policy tweak, etc is documented - I hope.)

I ran a quick scan of my (twiki) production wiki and found the following types (labels) in use:

find . -exec ls -Z {} \; | sed -e's/\s\s*/ /g'| cut -f4 -d' ' | cut -d':' -f3 | sort | uniq
httpd_sys_content_t
httpd_sys_script_exec_t
httpd_sys_script_ro_t
httpd_sys_script_rw_t
httpd_unconfined_script_exec_t
sbin_t

On my experimental system, it looks not-quite the same:

find . -exec ls -Z {} \; | sed -e's/\s\s*/ /g'| cut -f4 -d' ' | cut -d':' -f3 | sort | uniq
bin_t
httpd_config_t
httpd_sys_content_t
httpd_sys_rw_content_t
httpd_sys_script_exec_t

Which is only to say that the number of labels is manageable. For a production version, one would map the label in MANIFEST to the system-specific label via configure. E.g. MANIFEST says "perl_module_t"; configure says (on this fedora system) "perl_module_t => httpd_sys_script_ro_t". We don't have that many types: executable scripts, topic data, perl modules, temp/working data, config data.

I'm also keeping track of the boolean policy variables required for my new install. So far, the list is:

semanage -i - <<EOF
boolean -1 httpd_can_network_connect
boolean -1 httpd_can_sendmail
boolean -1 httpd_enable_homedirs
boolean -1 httpd_read_user_content
boolean -1 httpd_unified
EOF

It's probably not optimal, but this is not a production machine - selinux wasn't my focus.

So far, there are relatively few policy tweaks - but this will probably grow as I get more plugins running. But so far (and this includes Task FW support), the type enforcement file is (just):

policy_module(localWiki, 0.1.0)
require {
 type httpd_t;
 type httpd_sys_script_exec_t;
 type port_t;
 class lnk_file { read getattr };
 class tcp_socket { name_bind };
}
# Follow symlinks to Daemon
allow httpd_t httpd_sys_script_exec_t:lnk_file { read getattr };
# Web-started Daemon port listeners
allow httpd_t port_t:tcp_socket name_bind;

I certainly agree that we should make it easy to have a selinux environment for foswiki developers. First I've heard of "vagrant". I do run under VirtualBox, but with fedora. I guess it should be straightforward for someone with vagrant knowledge to setup a fedora distribution. The fedora install is simple; attach the ISO image, boot, follow the gui, wait for the updates and you have a working system. The only trap I've hit is that it does need about 1GB of RAM for the install - can be shrunk after, but I don't. If someone (Olivier?) can provide a generic recipe (for building a base OS for Foswiki), I'm willing to try building a kit. I'm not a Ruby, Chef or Puppet person...

Anyhow, hope this is a useful opening round of discussions. I'll try to remember to add more once I've released a TFW baselevel, but feel free to ping me if I forget.

-- TimotheLitt - 28 Jun 2011

Oh, I did a quick search on vagrant and fedora, and turned up https://github.com/walkah/veewee. Looks like building a fedora 14 base box has already been scripted... (current release is 15, but 14 should be good enough for our purposes.)

-- TimotheLitt - 28 Jun 2011

Hmmm, no nibbles on a Foswiki vagrant build for fedora?

I have a fleshed-out post-installer for the task framework that could be a base for selinux support. (As described earlier, plus evolution...). I believe this broadens the topic from "security checklists" to "Security automation and checklists". But this is a result of previous discussion being redirected here...

I've attached a snapshot Snapshot of Task Framework (post)-Installer & sample MANIFEST mapping file for fedora. It does a bit more work (it's main purpose is to setup the various softlinks needed for the task framework to autostart), and currently only processes the TFW's MANIFEST. (But there's if( 0 ) code to process every MANIFEST in the installation treee.) Note that it is designed to work on both the old and new wikis automagically.

The scheme works roughly like this (there are more details in the HELP text):

• MANIFEST is augmented with comments containing:
  • abstract selinux types (the interesting part of a security context label)
  • abstract file ownership (e.g. webserver)
  • directory data (same as files, but directories don't seem to be in traditional MANIFESTs)
• MANIFEST "mapping" files are defined to turn the abstract types into installed types.
  • selinux abstract types like wiki_perlmod_t => fedora's httpd_sys_script_ro_t:s0
  • ownership like webserverUser.webserverGroup => apache.apache
  • directory permissions like 0755 => 02755
  • file permissions like 0644 => 0464 (e.g. where access is by group, not owner)
  • File and directory permissions are further modified by "umask"s
• Mapping files are not required unless SeLinux support is enabled, but are useful in most circumstances
• The mapping file used is the first found in this list:
  1. Command line -m file
  2. Site policy mapping: lib/MANIFEST.map.LocalSite
  3. Kit (e.g. this plugin): lib/{Plugins/Contrib/..} Distribution.version file: lib/*iki/Plugins/TasksPlugin/MANIFEST.map.fedora.15
  4. Kit (e.g. this plugin): lib/{Plugins/Contrib/..} Distribution file: lib/*iki/Plugins/TasksPlugin/MANIFEST.map.fedora
  5. Wiki distribution.version file: lib/*iki/MANIFEST.map.fedora.15
  6. Wiki distribution file: lib/*iki/MANIFEST.map.fedora
• "distribution" is the os name returned by $^O (e.g. solaris, VMS), with linux special-cased due to the distribution-specific security variations.

For the (human) installer, this means that the security policy of the wiki can be changed to that of his site in exactly one place. Copy his distribution's map file, edit and save as map.LocalSite. Run the (currently post-) installer once, and everything updates. Thereafter, (s)he would expect addon-installations and wiki upgrades/updates to obey the policy.

And the pain and suffering caused every time a plugin is installed and destroys data/'s permissions (etc) vanishes!

If our default policies are OK, there's nothing special required - the installer (software) will "do the right thing" for supported distributions - and do no worse for others. Anyone can add a mapping file for distributions that we don't currently support.

For developers:

• MANIFEST files become important in the field, not just in our internal processes
• Additional data needs to be added, especially for SeLinux support - but this can be defaulted/automated
• We can phase this in as each MANIFEST is touched - though it would be ideal to have a focused work period so we can have a release with as much support as possible.
• This is all backward compatible - the additional data is just comments to older releases; and the installer should work with older releases.
• The code probably wants to be merged with some existing installation utility.
• We'll need to identify owners/champions for keeping the default mapping files for each distribution up-to-date
• We'll need some testing infrastructure.

In any event, that's what I've accomplished so far. I think it's a reasonable base for finally addressing the installation permissions/policy nightmares. And (to re-tie it to this topic), should reduce the "Security Checklist" and installation's on-going efforts to something more manageable.

The code (and sample MANIFEST/map files) are a development snapshot, so they will probably evolve some more. But feel free to look at them as concrete examples of the scheme, and experiment.

As always, thoughts welcome. (But note that this would take commitment from the entire team to implement and keep going in the mainstream.)

-- TimotheLitt - 23 Jul 2011

It's elsewhere in the development topics (See UseEnhanceAndMoveSpecFiles). I'd like to see a md5 hash added to the manifest so that the extensions installer can determine easily if files have been modified from the previous installation.

Oh... and to be cross-platform, is there any chance to implement your above proposal in Perl? bash scripts are not portable, and I suspect any core features restricted to bash would gather some objections.

-- GeorgeClark - 23 Jul 2011

The old bash script was ugly, slow and inadequate. It's not on the table.

If you look at the attachment, you'll see that the post-install script is a perl script! I don't have all the platforms to test, but the design is portable.

The mapping files are also perl config files (done files).

MD5 hashes are a reasonable idea, but orthogonal.

-- TimotheLitt - 23 Jul 2011

Oops - I obviously didn't review the attachments. Note that it is fairly simple to extend configure. I found the changes to add a "Test Email" button were very straight forward. Would it be possible / practical to run the check/set permissions under the control of configure? If so,

• Could the implementation be changed to be a perl Module under Configure. Foswiki::Configure::Permissions or something like that.
• Foswiki::Configure::Package call it to set the permissions during extension installation.
  • This runs under the control of configure, or from the shell under an extension installer, or the generic tools/extension_installer
• Add a "Check" or "CheckAndFix" button to Configure that runs a scan of the entire install.
  • (Can the spec files cover files that are not in the manifest? New Webs / Topics, etc. that were created by the wiki users?)
  • If so, this proposal could replace or supplement the checkTreePermissions currently done by Configure. The current implementation adds considerable overhead to every configure transaction, and would be better to change to an on-demand implementation.

I agree MD5 hashes are orthogonal, but any changes that impact the manifests have the potential of causing pain to all extensions, impact backwards compatibility, etc. So best to get them all on the table. The proposal to enhance the Spec files is an accepted proposal, so there is an opportunity to merge the requirements.

-- GeorgeClark - 23 Jul 2011

Maybe, but it would be tough. The problem is that SeLinux won't let the webserver access anything that isn't tagged for access. It can't say "is/should be" because it can't access the "is" - and the attempt will be logged and cause various analysis utilities to run. Similar problem with ownership & initial permissions - most of the setup has to be done with root permissions - which webservers don't have. Setuid would bring other objections to the table. So it isn't at all obvious how to break that circularity. Better to have it be part of an installer. As an installer, it could open the archive (tar/zip) files, extract & set permissions efficiently...

If, at initial install, directory permissions are set properly, topics should be covered thru inheritance. MANIFESTs current explicitly don't support wildcards - but that could be an extension.

Check and fix (equivalently, apply updated policy) was envisioned - but again, I don't see how to do it in configure since configure is in the protected/constrained/restricted environment of the webserver. Agree that much of the configure checking would be better done on demand - why can't this be a SUBMIT button?

So far everything I've done for this has been back and forward compatible with existing MANIFESTs. But yes, there will be some effort (pain seems too strong) to get all the extensions to conform. I would think that MD5s would be pretty transparent - don't require human judgement, just add to the kit builder and install script. The only tricky part is making the MANIFEST addition compatible - I suppose a ##C <md5> <path> record could do this fairly painlessly. Since it would be added (or updated) by the kit build script, the impact on people would be zero (especially if all the ##Cs were at the end of the MANIFEST to minimize visual clutter when editing.) (C is for Checksum.)

Maybe there's room to merge with the specfile work - but conceptually, spec files say how to configure something vs. MANIFESTs which are an inventory of what/how to install (and should include the spec files)...

It is worth looking at the attachments - the script has fairly long help text at the bottom, and the sample manifest and map files make the bulleted lists more concrete.

Since this started as Paul's topic, hopefully he'll chime in...

-- TimotheLitt - 23 Jul 2011

Unfortunately I suspect that selinux removes quite a bit of the utility of the web configure - Installing and updating extensions would be prevented I assume. However a lot of that can all be done from the cli, and Sven has been asking for a shell based configure tool which could work around the update restrictions. We probably need to detect certain configurations - SELinux (or other MAC) Andrew File System, and debian package based installs and turn off the pieces of configure that would need root permissions.

* Hm.. that reminds me - Maybe we need to break out the actual permissions: Permissions::SELinux Permissions::AFS, Permissions::Windows etc. as I recall there were complaints that the simple octal permissions is also meaningless when AFS is in use.

Splitting out the checking / reporting from the actual updates would permit some utility from the webui. And for sites not running selinux, file permissions can still be set, so there would be some utility for what is currently (assumed) the majority of the install base. Foswiki::Configure::Package is the installer, and is usable from both the bin/configure web script, and the cli. So integrating it into Package.pm is adding it to our current installer.

The missing pieces are a bootstrap - how do I run the installer before Foswiki is installed. And packaging - generating "package" files that can update the core from under the control of Package.pm. (That's been done experimentally / manually but needs to be automated). It ties in with needing a way to release smaller incremental updates, with simple web-based installation. ... for sites that are not running SELinux anyway. Package.pm pretty much 100% supports it already. It needs some tweaks to prevent "uninstall" of a core package, substituting a "restore backup" proabably. But it can already install any file in the distribution, even if the installation has renamed webs, topics and directory locations. All of that gets mapped by Configure::Package and Configure::Util.

And regarding MANIFEST changes. There is a "MANIFEST2" format currently unused but is supported by Package.pm - Version 2 of the embedded MANIFEST file includes the MD5SUM per file. The installer supports it, but BuildContrib doesn't generate it.

I think the plan for UseEnhanceAndMoveSpecFiles (and from some other discussions) was to drop the self-executing code from the PackageName _installer files, and rework them to become a Spec file - with the MANIFEST, DEPENDENCIES, and other Spec information needed for installing and configuring the Extension. (and hopefully Core too). One huge issue we have with the current _installer files is that they tie us to the tools/extender.pl. It makes it very difficult to improve Package installation, since every package ever built has a dependency on the extender.pl API. It really needs to go.

Unfortunately some of this stuff is boiling the ocean. But our current Extensions handling is in desperate need for improvement.

-- GeorgeClark - 23 Jul 2011

And a note to Paul - some of his initial needs were more centered around configuration of Foswiki, not the file system. I have another stalled proposal - AddConfigurationWizard - that was to address at least part of his concerns. Chose a Foswiki deployment scenario, and have it configure things appropriately. Probably better done in JavaScript, so it auto-sets the relevant configure settings for a deplyment strategy.

-- GeorgeClark - 23 Jul 2011

SeLinux isn't that bad once it's running. It's the bootstrap that's really hard. Context tags are inherited from the directory by default. And that covers most of the cases. As long as executable scripts are in bin (tagged executable) and perl modules in lib/areas tagged script read and r/w data in data/pub, the selinux parts ought to be managable. There are probably a few exceptions to make life interesting.

Quick primer: SeLinux permissions are orthogonal to chmod permissions. To access a file, you need to pass the octal tests, and the SeLinux policy. The policy applies roles to actors and contexts to objects; it then defines the permissible interactions. So there's a policy that says "when an executable labeled httpd is run, it's allowed to do the following (and nothing else.)" It then says things like "can create files in directories labeled with the httpd_script_rw_t" and execute (as cgis) scripts labeled httpd_sys_script_exec_t. And so-on - the polices are very detailed. There are also some knobs that the policies will obey - stuff like "can apache access a home directory." The policies originated at NSA - distribution independent - but the distributions that contain selinux customize/improve them. Fedora has developed policies for 100s of packages, and they are enabled by default for all the daemons. (Relevant here including httpd, sendmail, cron, clamav, nfs, iptables, and init scripts) Since fedora feeds RHEL, this will impact many sites' out-of-the-box and ongoing experience. SeLinux takes some getting used to, but it seems to be being tamed - I think of it like a firewall for the host. Google for more complete descriptions. And yes, ACL-based systems tend to have similar architecture (octal permissions + something else).

Back to the subject: I've never tried to install from the GUI because the current mess corrupts everything (even octal permissions and file owers) - the tar files contain directories and as soon as you untar anything, your whole installation is dead. (Because lib and data and usually pub are in every extension's .tar file, and the untar changes their ownership & permissions to ones that don't work on fedora.) No idea how others cope in the real world. SeLinux only adds another twist. I find myself having to list an extension's tar file at the CLI, manually build a tar command with cut and paste that excludes existing directories (but adds new ones) to unpack the kit, then chown chmod and chcon subsets of those files to make them accessible. At least that way, the mere act of unpacking a new extension doesn't shut down the existing webs.

Since one generally wants files to be owned by the webserver, it (GUI) ought to be able to unpack. Kits currently come with very permissive permissions on the files, so chmod to more restrictive permissions should work. And once a file has a security context (tag), moving the file to a new directory does not change it; the tag moves with the file. So if we had an unpacking directory with each possible context available, we could extract files to that directory (tagging them), then move (mv/rename) them to their proper destination. Same trick with new directories. (Necessary only when the default context of the target directory doesn't match the needed context for a file.)

All this, however, is another level of complexity, convenience and effort. I was starting from the position that if we could do something within the current framework at the CLI level, things would improve dramatically. Maybe I could try a new plugin or install a patched one in less than an hour... and without dread.

The scheme I outlined allows that - and provides the metadata for a more ambitious scheme like the one you're outlining, when resources and energy allow.

I'm a bit leary of wanting to bite off too much at once - note that the task framework, which was quite useful when it was a dozen lines of plugin garbage collection, is now 30 perl modules and ~6500 lines of (non-comment) code. And not quite done But as I said in that context: he who does the work gets more votes. For these issues, I promised to provide my post-installer - done. And that I'd help with selinux - still willing. But I'm not on the hook for fixing all the packaging/installation woes. So the opinions here are those of a user/admin with a clue - not a committed developer. (This thread came out of my 'first experiences' topic - and I was asked to move the file permissions/selinux issues here...The feature proposal is still Paul's.)

I'd really like to see incremental improvement rather than an ocean-boiling exercise...

That said, of course I care and will provide input as long as its welcome...

-- TimotheLitt - 23 Jul 2011

For what it's worth, this proposal isn't "mine" until I put myself in the CommittedDeveloper field As it currently stands, I am running our production site from trunk. Using pseudo-install, and git. So I am possibly the least in touch with the average admin experience.

I do know that it's ugly (even embarrassing, compared to modern CMS/wiki platforms). So your input is certainly welcome. And so I wanted this to be a place where we can hammer out a spec on what a more admin-friendly Foswiki might look like. But I'm not even sure where to begin (hopefully with the low-hanging fruit first, right?).

My workplace has me working hard on performance/unicode/wiki-app features/UI, so I'm afraid I can't be the one to work on this, any time soon

-- PaulHarvey - 25 Jul 2011

See also:

• Support.Question934
• Support.Question744
• Tasks.Item11148

-- PaulHarvey - 04 Sep 2011

I attached the latest snapshot post-installer in case it's of interest. It has a few more features to make it easier to selectively customize site (and machine) policy at the file level, as well as some packaging magic to allow MANIFEST to be authoritative for (my) kits. Integrating something along these lines seems like a place to start.

-- TimotheLitt - 28 Sep 2011