Item380: Install the Public Cache system on foswiki.org

To alleviate performance problems on foswiki.org, install on it the http://foswiki.org/bin/view/TWiki/PublicCacheAddOn

Tests made on http://cache.foswiki.org seem ok. I will install it Wed 03 Dec 2008 at lunch (Euro time) normally.

This will imply implementing the fixes (to the templates mostly) detailed on the above page, notably:

• templates/viewtopicactions.nat.tmpl for the Main.YourAccount link instead of "Log in", specifically, $login replaced by (space)Main.YourAccount
• data/Main/YourAccount.txt file added
• html comment * Set PUBLIC_CACHE_EXPIRE = -1 added to
  • WebHome
  • MyItems
• data/TWiki/TWikiWebTopicActions.txt set restrictedactions value to empty string
• /usr/local/etc/apache2/Includes/cache.foswiki.org.conf last RedirectMatch line wrongly redirected to Main instead of Home

FAQ: Once in place, if you see problems, here are workarounds:

• to clear and reset cache, go to http://foswiki.org/bin/pcad and press the clear button on the second bullet line
• to make a page uncacheable, set in it a line * Set PUBLIC_CACHE_EXPIRE = -1 See complete syntax on http://foswiki.org/bin/view/TWiki/PublicCacheAddOn#Syntax
• try to rewrite pages having parts displaying differently for different users has a cached fixed content page and a link to an uncached one if the uncached part is not something important
• in case things go haywire,
  • the cache can be disabled for you (your IP) by going to http://foswiki.org/bin/pcad and click on bypass (4th bullet) for tha amount of time you want
  • the cache can be totally disabled by logging root on the server and typing:
    cd /home/foswiki.org; ./twpc/uninstall .
    the cache can be later re-enabled by a: cd /home/foswiki.org; ./twpc/install -u .
• if something reaaaaally bad happens, send me a SMS or phone call to my mobile phone that I will leave to the admins in #foswiki-admins
• The bin/view is now a C binary, callable only by CGI. Thus you cannot call it from shell anymore. A workaround for it is now from scripts is to call it as
  if test -e vief; then perl ./vief; else perl ./view; fi
  instead of just perl ./view

-- ColasNahaboo - 02 Dec 2008

Installation performed on Wed Dec 3 11:57:07 UTC 2008

Steps performed:

• templates/viewtopicactions.nat.tmpl for the Main.YourAccount link instead of "Log in", specifically, $login replaced by (space)Main.YourAccount
  old version saved as templates/viewtopicactions.nat.tmpl.orig
• data/Main/YourAccount.txt file added (copied from the cache. instance)
• html comment * Set PUBLIC_CACHE_EXPIRE = -1 added to
  • WebHome
  • MyItems
• data/TWiki/TWikiWebTopicActions.txt set restrictedactions value to empty string
  old version saved as data/TWiki/TWikiWebTopicActions.txt.orig
• /usr/local/etc/apache2/Includes/foswiki.org.conf untouched, as the bad RedirectMatch line was already commented out
• Cache installation, from version 4.7 of http://hg.colas.nahaboo.net/twiki-colas/twpc/summary
  • wget -O twpc-4.7.tgz http://hg.colas.nahaboo.net/twiki-colas/twpc/archive/3a256cf76f24.tar.gz
  • tar xfz twpc-4.7.tgz; mv twpc-* twpc
  • ./twpc/install -O -q -t0 .
• updated and closing this topic

-- ColasNahaboo - 03 Dec 2008

Issues

• there is the option to force the cache to create a "view" entry in the TWiki logs even on cache hits. I did not enabled it for speed, thinking that we will rely on apache logs & awstats instead. But we could enable it if you feel it is useful (for read stats on WebStatistics), for this do: cd /home/foswiki.org; ./twpc/install -O -t0 .
• now each time a page is edited the cache is immediately cleared. We could get a bit more CPU saving by having a grace delay and clear the cache after N minutes of no edits (N=17 per default), at the expense of only people having edited pages recently seeing the new version, which could be awkward when people mention pages on IRC to other fossies, who would see the old version. To enable this more aggresive caching, one should do: cd /home/foswiki.org; ./twpc/install -O .
• IMPORTANT if foswiki code is updated, the addon should be re-installed (to re-install patches to the code)

Fixes:

• 03 Dec 2008 Create a subskin for properly subclassing viewtopicactions.nat.tmpl
  • Done: as a sub skin nat_cached, thus a file templates/viewtopicactions.nat_cached.tmpl
  • Also created data/Main/TWikiWebTopicActions.txt to override and not change the default data/TWiki/TWikiWebTopicActions.txt
• 03 Dec 2008 installed now without -q ,to have logs, so we can measure the hit/miss ratio later
• 04 Dec 2008 fix: disabled the direct links to cache admin commands, that were triggered by crawlers (on my sites webs TWiki, Main, Sandbox are excluded from searches by robots.txt so I didnt see this problem before) crawling the PublicCacheAddOn page. The main problem was the build, generating building all the site page caches
• 05 Dec 2008 installed a new version that tries to provide a faster save
• ColasNahaboo - 06 Dec 2008 - issue found: date command is not the good one, the -r option is not the same. Disabling cache till this is fixed
• ColasNahaboo - 06 Dec 2008 - version v4.8 allowing to set the PATH installed at 14:00 (the server was nearly blocked by a yahoo! crawling. I installed GNU Coreutils 6.10 (Ubuntu uses this version) to get the GNU date in /usr/local/gnu. Cache installed with the options "-O -p/usr/local/gnu/bin -T300 -t60=

---

Is the caching now off? Because Item398 still persists.

• ArthurClemens - 06 Dec 2008