Item14126: Random topics or webs get access restricted

-- Ehj52N - 03 Aug 2016

Meta

I have a foswiki installation running with v2.1.2 with the following plugins:

Here is a list of the plugins currently installed and enabled in the wrench configuration on this Foswiki site:

• SpreadSheetPlugin (08 Apr 2016, 1.22): Add spreadsheet calculations like "$SUM($ABOVE())" to Foswiki tables and other topic text
• SlideShowPlugin (08 Apr 2016, 2.31): Create web based presentations based on topics with headings
• BreadCrumbsPlugin (23 Apr 2014, 2.51): A flexible way to display breadcrumbs navigation
• CommentPlugin (08 Apr 2016, 2.91): Quickly post comments to a page without an edit/save cycle
• CompareRevisionsAddonPlugin (1.114, 1.114):
• ConfigurePlugin (11 Apr 2016, 1.05): configure interface using json-rpc
• DBCachePlugin (09 Mar 2016, 9.20): Lightweighted frontend to the DBCacheContrib
• EditRowPlugin (19 Mar 2016, 3.315): Inline edit for tables
• FilterPlugin (29 Apr 2016, 4.23): Substitute and extract information from content by using regular expressions
• FlexFormPlugin (08 Mar 2016, 5.20): Flexible way to render DataForms
• FlexWebListPlugin (25 Sep 2015, 2.00): Flexible way to display hierarchical weblists
• GenPDFAddOnPlugin (1.3, 1.3): GenPDFAddOnPlugin helper plugin for GenPDFAddOn renders the %GENPDF% tag
• HistoryPlugin (1.13, 1.13): Shows a complete history of a topic
• HolidaylistPlugin (2.001, $Rev: 14328 (2012-03-15) $): Create a table with a list of people on holidays
• HomePagePlugin (1.23, 1.23): Allow User specified home pages - on login
• InterwikiPlugin (1.23, 1.23): Link ExternalSite:Page text to external sites based on aliases defined in a rules topic
• JQueryPlugin (10 Apr 2016, 7.04): jQuery JavaScript library for Foswiki
• LdapNgPlugin (23 May 2016, 6.20): Query and display data from an LDAP directory
• ListyPlugin (22 Sep 2015, 1.00): Fancy list manager
• MailerContribPlugin (2.82, 2.82): Supports e-mail notification of changes
• MetaDataPlugin (4.10, 4.10): Bring custom meta data to wiki apps
• MimeIconPlugin (17 Jul 2015, 1.30): Icon sets for mimetypes
• NatEditPlugin (06 Jan 2016, 9.07): A Wikiwyg Editor
• PreferencesPlugin (1.16, 1.16): Allows editing of preferences using fields predefined in a form
• RedDotPlugin (5 Mar 2015, 4.00): Quick-edit links
• RedirectPlugin (1.12, 1.12): Create a redirect to another topic or website.
• RenderListPlugin (2.28, 2.28): Render bullet lists in a variety of formats
• RenderPlugin (29 Apr 2016, 4.10): Render WikiApplications asynchronously
• SmiliesPlugin (17 Sep 2015, 2.03): Render smilies like smile as icons
• SubscribePlugin (06 Nov 2015, 3.5): This is a companion plugin to the MailerContrib. It allows you to trivially add a "Subscribe me" link to topics to get subscribed to changes.
• SyntaxHighlightingPlugin (1.21, $Rev: 9720 (2010-10-25) $): Highlights code fragments for many languages using enscript.
• TablePlugin (03 Feb 2016, 1.154): Control attributes of tables and sorting of table columns
• TagMePlugin (2.1.1, $Rev: 20120727 (2012-07-27) $): Tag wiki content collectively to find content by keywords
• TinyMCEPlugin (1.30, 1.30): Integration of the Tiny MCE WYSIWYG Editor
• TopicInteractionPlugin (17 Jul 2015, 4.00): Improved interaction with attachments and DataForms
• TwistyPlugin (1.63, 1.63): Twisty section Javascript library to open/close content dynamically
• UpdatesPlugin (1.01, 1.01): Checks Foswiki.org for updates
• WebLinkPlugin (1.31, 1.31): A parametrized %WEB macro
• WysiwygPlugin (08 Apr 2016, 1.33): Translator framework for WYSIWYG editors

Problem

I get random topics and webs with access restrictions. Hence, sometimes a single topic is not accessible for not logged in users and another time a whole web get's restricted.

Workaround

This workaround is not really good, because the topics and webs, that get blocked, are somehow chosen randomly.

1. Access the blocked topic or web as admin.
2. Open the according web's ~WebPreferences topic.
3. Edit the ~WebPreferences; not really edit something but just open in raw edit mode and save it.
4. Wait some time and the access set-up is fixed again.

Comments

Do you need additional information about the set-up or server environment?

-- Ehj52N - 03 Aug 2016

Are you using mod_perl or fcgi? For something to come/go like this, it sounds like there could be some in-memory reuse issues. You might try running with fcgi if on mod_perl. We use fcgi on foswiki.org and have not seen any inconsistencies like this. Rather than editing/saving, you might try reloading the server to see if that clears the issue. If that works, let us know.

-- GeorgeClark - 03 Aug 2016

Hi Clark,
I am using mod_fcgid. I tried reloading the server various times and it did not work. Today, I updated all web's WebPreferences following UpgradingFromOlderTWikiReleases#Convert_empty_DENY_ACLs_to_ALLOW_42_wildcards and updated them by hand. I hope that this solves my issue. I think that the problem here is that it must be clearer stated in the migration guide that the WebPreferences of each web have to be updated, too.
By the way, sorry for the late reply. It seems that the WebNotify creation script is not working well, as I cannot find my username on this topic 6 days after creating this issue.
Kind regards!

-- Ehj52N - 09 Aug 2016

The problem happened again and the workflow to open and directly save the according WebPreferences failed because the MySQL took more than 5 minutes to process this save. I think that the according cache update took so long. Hence, I disabled caching and the save works fast and without any error.

So, maybe the cache is the source of the problem!? What do you think?

-- Ehj52N - 06 Sep 2016

Yes we've seen some performance issues with the page cache, though mysql seems quite a bit better than sqllite. We've got some fixes planned for 2.1.3, which will reduce the size of the dependencies table. Also, if you have a lot of webs, the WebLeftBarWebsList will result in every topic becoming dependent upon every web preferences. Yet another reason to disable the poorly performing dynamic webs list.

-- GeorgeClark - 31 Oct 2016

Hi George, thank you very much for your update. Is there an option to replace the dynamic webs list by something more static? I do not want to update all WebLeftBar by hand, when adding a new web. Looking forward to version 2.1.3. Kind regards, Eike

-- Ehj52N - 04 Nov 2016

You could make a static WebList topic ... Main.CommonWebsList and then %INCLUDE it in place of the System.WebLeftBarWebsList. But this still means editing every WebLeftBar topic.

• In the webs we ship (Main, System, Sandbox and Trash), the WebLeftBarExample has an INCLUDE for WebLeftBarWebsList, so you would create the WebLeftBar which will override the WebLeftBarExample.
• In any other web taken from the _default Web, it just has a WebLeftBar topic, which also has the %INCLUDE for the webs list.

A quick solution (we do not recommend!) is to simply update the WebLeftBarWebsList topic. It avoids changing ever WebLeftBar, but you would lose your changes the next time you upgrade foswiki.

The problem with the WEBLIST macro has been well known, but there isn't really a good solution for it. In order to generate a weblist, the Foswiki core must:

1. Read every WebPreferences file to check if the current user can view the web,
2. Read every WebHome to check if the current user can view the topic.

And since it's in the WebLeftBar, this happens for every page view in your wiki. A lot of webs means a lot of auth checking. If you use the PageCache, then it means that every cached topic has a dependency on every WebHome and WebPreferences topic in your wiki.

Unfortunately no matter how you look at it, it's an expensive feature.

-- GeorgeClark - 04 Nov 2016

Best would be to remove the WEBLIST from the sidebar and have the list of all available webs on the frontpage. Anyway, each topic will always depend on the preference system reading required WebPreferences along the rendering path. There is not much we can do about this fact other than ignoring dependencies partially ... which could result in caching errors, of course (stale content that should have been rendered differently due to content changes).

There are other inefficiencies in the page caching code that we won't be cleaning up until after 2.1.3 has been released.

Let's please keep these performance issues aside from the original report made here. Is there any more insight in what caused topcis and webs getting wrong access rights sporadically, Ehj52N? Or is it in fact related to the page cache being activated or not?

-- MichaelDaum - 12 Dec 2016

Setting this task to No Action - No feedback, and no further reports. Please reopen if the problem can be reproduced.

-- GeorgeClark - 18 Feb 2017

Commenting is disabled while not logged in