Feature Proposal: Prevent redundant file reads in store

Motivation

Investigating the most expensive calls in Foswiki-2.0.0 using nytprof shows that Foswiki::Store::Rcs::Handler::readFile() is called up to 18 times for the same file (PatterSkinNavigation) and overall 118 files when visiting System. That's quite a lot. Disk io is THE major bottleneck unless you are using SSDs.

There is a patch branch implementing this.

Description and Documentation

See also Tasks.Item13525

Examples

Impact

WhatDoesItAffect:

Implementation

-- Contributors: MichaelDaum - 27 Jul 2015

Discussion

Pushing this out to 2.2, a number of the issues reported here have been addressed in other patches.

-- GeorgeClark - 06 Dec 2015

I'm adding a concern, in that this doesn't really have a design to comment on. Many of the duplicate I/O's were fixed earlier when it was determined that the Meta Cache was not caching. With that fixed, we need to decide whether yet another layer of caching in store is the right thing, or if more use can be made of the Meta Cache. The Meta Cache seems to be only active in the search code. It avoids the significant overhead of de-serializing the topic meta in addition to avoiding extra I/O.

-- GeorgeClark - 20 Mar 2016

Most of the issue has been resolved in 2.1.9. See Item15206 and Item15338. The MetaCache has been refactored to make sense; a fast preferences backend has been implemented. Both together significantly lower disk io per click.

-- MichaelDaum - 14 Oct 2024

Commenting is disabled while not logged in