Performance Supplement

Foswiki Performance Considerations

Server based optimizations are good and might be necessary but may not be the final bottleneck. Once done the client side communication between the browser and server will become central.

Analysis

An easy method to get some reliable data about Foswiki performance from an end user perspective is by using tools which measure the time to build up the called page. Most browsers have request timeline functionality in their onboard dev tools.

Another good tool to analyze your site's overall performance is the Yahoo YSlow tool or Google's PageSpeed tools. These will do an analysis of all the requests made on a page and report information about steps that can be done to improve performance. Some of the recommendations are very applicable to Foswiki, others not as practical. They are also a good tool to determine if server configuration options like Expires tags and compression are actually active.

( Some words about Benchmarking possibilities) Here are pages listing tons of related tools:

• www.opensourcetesting.org
• www.softwareqatest.com

An analysis about potential impacts on the page load time:

• Optimizing Page Load Time

Server Optimization

Persistent Perl

On the server side the FSA architecture is the foundation for speed optimizations. There is ongoing work to further improve it. As a result the following extensions will profite from this architecture and are the base to reduce Foswiki overhead:

• ModPerlEngineContrib
• FastCGIEngineContrib

These Extensions permit the web server to load and keep the compiled perl code in memory. In conventional CGI, all perl modules are loaded and compiled for every web request. In addition, Foswiki will attempt to load and retain certain static information in memory but that is only helpful when the information can be used for multiple web requests.

Ex. The default HtPasswdUser module for managing the list of users loads and caches the contents of .htpasswd. For very large .htpasswd files, this can save very significant time per transaction.

Either of these extensions can deliver a significant performance improvement for Foswiki. However there are some key configuration changes that should be done if using Persistent Perl. The operation of "Forking to a new thread" is extremely costly when used with Persistent Perl. So Forking should be avoided where possible:

• The Foswiki {Store}{Implementation} should be changed from Foswiki::Store::RcsWrap to Foswiki::Store::RcsLite
• The Foswiki {Store}{SearchAlgorithm} should be changed from Foswiki::Store::SearchAlgorithms::Forking to Foswiki::Store::SearchAlgorithms::PurePerl

Database Considerations

Since a default Foswiki installation uses a text based database (flat files), it is rather necessary to have local disks with good response times instead of another round of network based calls. The search speed is considered to be good enough for several thousand of topics but becomes a bottleneck when using 10,000+ files.

The following contribs are available to speed up access to the data base especially for search operations:

• DBCacheContrib
• DBCachePlugin

Caching Macros

This Plugin caches Foswiki macros in selected topics for faster page rendering. Page Caching is built-in on Foswiki 1.1. However effective use of the cache requires use of Persistent Perl.

Other Apache configuration for performance

Apache Module Expire

This module will completely avoid a requests for files from the browser for any files marked with a time stamp for validity. Depending on configuration the length of validity can be configured but even 1 day or just a few hours will have an effect. Pictures are usually pretty static and do not need to be re-verified regularly, so they are ideal candidates for mod_expire. Certainly any attachments in the System web would greatly benefit from a far future expiration time. Module expire will guaranty that the browser is not even asking on a normal page call to re-check if the file might have been changed within the valid time period. This reduces the amount of basic calls and has a higher effect if response times are longer.

The ApacheConfigGenerator provides sample rules to set the Expires tag.

Apache Module Deflate

Since computers are pretty quick, the CPU overhead caused by compression on server side is usually much less compared to the advantages gained by sending small files over the network. This is "normally" even the case when using encryption on top of compression. Hence enabling compression should be considered and makes sense with HTTPS as well.

Other Server Side Considerations

VarWEBLIST overhead

WEBLIST can be quite slow. See BestPracticeTip20 for advice on replacing WEBLIST with a topic holding a static version of the output from this macro.

Session Expire After

Consider setting {Sessions}{ExpireAfter} to a negative number while also enabling the session clean up script tools/tick_foswiki.pl via a cron job. Please find more details in the description for {Sessions}{ExpireAfter} which is part of configure.

TOC Page Reload

If the page is fully reloaded while just using the %TOC% macro, be sure that the configuration parameters: $Foswiki::cfg{ScriptUrlPath} = ''; and $Foswiki::cfg{ScriptUrlPaths}{view} = '' are set correctly. See Support Question 324 for more details.

Extensions

(Have some general statements about how the amount of Extensions influences the general Foswiki performance. Put a table listing Extensions which speed up and Extensions known to operate not really quick or to cause speed issues (TreePlugin, HolidaylistPlugin, etc.) in certain situations/combinations.)

For some extensions possible performance issues should be considered.

• Tree Plugin Perfomance

Load Caused by Spiders

• ManagingServerLoadFromSpiders

Client Optimization

Modern browsers are all using a cache which holds data in the memory but also on the local disk. This cache should be active and not turned off. It will also cache data retrieved via HTTPS (Firefox 2.x had issues with it) unless this is actively prevented due to special security considerations.

To even further minimize calls and the necessary data amount the apache modules mod_expire and for compression mod_deflate should be active. Module expire can e.g. take care of pictures, javascript and css files, mainly static files which are changed less frequently. Module deflate then could handle everything which is not already compressed. Both modules are worth to be active in most cases.

Pipelining

• Firefox pipelining

Extensions

For the Client side the BrowserBoosterPlugin is available to improve performance in case of higher network latency or to work around Firefox 2 issues. ( Note: This extension is not needed with current versions of Firefox.)

Content Optimization

Search Optimization

Often performance issues on isolated topics can be addressed by subjecting the %SEARCH macros to careful review.

• Look at the "bounds" of the search. Restricting a search to a subset of topics scope="topic" topic="*Template" can provide a significant boost.
  • Use of the QuerySearch "name~'*Template'" is not sufficient. This still causes significant topic I/O. Adding the topic= bounds on the search reduced some searches on foswiki.org by many seconds.
• Be cautious of nested searches. (Embedding a %SEARCH inside the results of an outer SEARCH). Although extremely powerful, they can have a significant impact on performance.
• Rewrite older regex searches into the new QuerySearch language
• Look into extensions that index content topic: SolrPlugin, and once available, MongoDBPlugin

Identifying topics to optimize

Apache logging can be modified to log detailed information about each query, and makes it easier to identify topics that should be targeted for review. Some things of special interest:

• Total number of bytes actually transferred to the end user. Very large responses might benefit from search paging.
• The process id that handled the request - for possible correlation to system task information
• The completion status - did the query complete or fail, or was the query abandoned by the user. (Long running queries with X status indicates wasted processing as the user gave up.)
• The actual run-time of the query

The following log format can be used to write performance information for import into a spreadsheet or database. The options are defined below:

LogFormat "%{pid}P,%{%Y/%m/%d-%H:%M:%S}t,%a,%u,%T,%X,%s,%O,\"%U%q\"" perf

%{pid}P

The Process or thread ID of the child that serviced the request $ %{%Y/%m/%d-%H:%M:%S}t: Date-Timestamp when the request was received

%a

Remote IP Address

%u

Remote user from auth

%T

Time taken to serve the request, in seconds

%X

Connection status when completed X = aborted, +, may be kept alive, -, may be closed. (If X, then the user abandoned the request)

%s

Status: if internally redirected, the original status

%O

Bytes sent including headers - requires mod_logio

\"%U%q\"

The complete URL Path and Query string

It might also be useful to add

\"%{Referer}i\"

The page containing the link followed to this location

\"%{User-agent}i\"

The user agent, which can identify bot traffic,.