Feature Proposal: Allow users to customise System topics (such as WebSearch) in an upgrade safe way

Motivation

Allow admins and plugins to easily customise system topics.

Description and Documentation

I want to change the following topics to be built up using templates:

• WebSearch
• WebSearchAdvanced
• WebChanges
• WebTopicList
• WebIndex

I will also add a "How to customise system topics" FAQ in the Support web.

Examples

WebCreateNewTopic is already created in a similar way.

Impact

This will not impact on people upgrading from Foswiki 1.0.x, or from TWiki.

Implementation

Here is an example of the implementation steps for WebTopicList:

(1) Change WebTopicList.txt to:

<!--
  * Set VIEW_TEMPLATE = WebTopicListView
-->

(2) New WebTopicListViewTemplate -> added to standard distro

%TMPL:INCLUDE{"view"}%

%TMPL:DEF{"content"}%
%TOPICLIST{"   1 [[%BASEWEB%.$name][$name]]"}%
%TMPL:END%

If someone wanted to customise WebTopicList, they would create a topic called MySkinWebTopicListTemplate and add myskin to the skin path.

If a plugin wanted to ship a new WebTopicList, they could ship a MyPluginWebTopicListTemplate and a user will add myplugin to the skin path. This would not be a possible using AddDefaultTopicParameterToINCLUDE.

Another advantage to templates is that we can re-use parts of the default page. For example, we might want to change parts of the search form but still inherit some the majority of it from the default.

-- Contributors: AndrewJones - 16 Mar 2010

Discussion

Concern raised as there is no clear spec yet so we do not know what we accept.

-- KennethLavrsen -- 04 Sep 2009

Refactored so hopefully the spec is now clear.

-- AndrewJones - 04 Sep 2009

This proposal could also make AddAutoViewTemplatePlugintoFoswiki even more useful.

-- AndrewJones - 06 Oct 2009

ok, I think what Kenneth is trying to point out, is that this topic is not a work specification yet - it is a problem specification, but we can't really consider it until there is a suggested implementation idea.

I think there is an easy possibility - using something of the =%INCLUDE{"MAINWEB.WebChanges" default="SYSTEMWEB.WebChanges"}% but this only addresses one level of customisation - and I can say from experience that there are often more levels needed .

-- SvenDowideit - 07 Oct 2009

Added pointers where we use this approach already. Added some more topics that need the same treatment.

-- MichaelDaum - 07 Oct 2009

I think Sven is onto something there. Consider this: rewrite the %INCLUDE{"%SYSTEMWEB%.XxYy"}%'s to %INCLUDE{"%MASTERWEB%.XxYy" default="%SYSTEMWEB%.XxYy"}% and define MASTERWEB, by default, to be %SYSTEMWEB%. MASTERWEB is not finalised, which means you can redefine it locally. If you want to specify a different web for a subset of your master topics, then the default takes over for those missing from the subset. If you want a multi-level customisation, you can opt to define all the master topics in the first subset web as includes from the system web.

-- CrawfordCurrie - 07 Oct 2009

I believe the %INCLUDE{"%MASTERWEB%.XxYy" default="%SYSTEMWEB%.XxYy"}% is

• Generally a very useful feature
• A feature we can use for ALL the cases where we want a default include which we distribute and enable the user to define his own.

If we can agree to this step the next is to find a COMPATIBLE path to how we go from 1.0.X to 1.1.0.

And I see 3 possibilities (and in reality 2)

• 1A The MASTERWEB approach where the user has to either define or create a master web.
• 1B Always use the MAIN web. If we define a MASTERWEB as Main in DefaultPreferences we enable both 'always Main' and 'user defines'. We use the Main web approach today for things like the default user topic template.
• 2 Use a naming convention. Today the PatternSkin has implemented a naming convention where a number of topics are suffixed ...Example. I find the suffix naming OK but not great. Renaming it to ..Default would be more intension revealing.

I would like to see the INCLUDE enhanced as suggested.

And I would like that we do the default approach the same way for all features. Ie. we either implement 1 or 2 but do it the same way for all the features.

The challenge is how to make the 1.0 to 1.1 transistion compatible. Ie. avoid an upgrade hell going 1.0 to 1.1.

I have refactored the proposal above to where we have come so far.

-- KennethLavrsen - 07 Oct 2009

Thanks for the discussion and refactoring.

I like the INCLUDE idea. When this was first proposed, some on the mailing list suggested the templates were too complicated. This should appease them, while still allowing advanced users and skins/extensions to use templates to give more control.

I also agree that ...Default is a better naming convention. We could change PatternSkin so that its consistent.

Do we then need to change those that already use templates (such as WebCreateNewTopic) to be consistent with this approach?

With regards to upgrading from 1.0.x, it should be invisible. If users have already made changes to the System topics, they already expect it to be overwritten on upgrade anyway (at the moment I keep a list of topics that I have changed and manually restore them).

I will make some changes locally tonight so I can test any impact on upgrading from 1.0.x.

-- AndrewJones - 07 Oct 2009

One thing about the Main web I do not like is it's unclear purpose. It's for users and groups as well as for some customisations like prefs or templates. Both have nothing in common and the name neither reflects the user nor the customization part.

Why not have a dedicated web for those things?

-- CarloSchulz - 07 Oct 2009

I think we all agree that Main is a poor name. We have agreed that since year 2000 I think. Problem is that it is too late to change now. People have hardcoded Main everywhere in signatures, in applications - everywhere. It is a nightmare to even think about changing this in an old installation. On top of that is retraining people.

And adding yet another web to hold maybe 5 custom topics is not something I would want. Having a System web, a Main web, and a Sandbox is more than enough IMHO. The advantage of the MASTERWEB setting defaulting to Main (or rather defaulting to USERSWEB) is that if YOU want to have a new web for default topics all you need to do is create this web and redefine MASTERWEB in SitePreferences. It will take 3 minutes to do for those that want this and be safe for all later upgrades.

If holding the default topics in Main is a concern Carlo, maybe you are more leaning towards keeping this in System web and using the suffix naming convension. I am beginning to lean towards this too as this web is easy to make a Sysadmin only area and keep these system admin type of default documents away from Main web.

-- KennethLavrsen - 07 Oct 2009

Yeah I think the default topics should be in the System web, with customisations made in the Main (or the MASTERWEB) web.

-- AndrewJones - 07 Oct 2009

Main is an obvious candidate, because it already holds a mass of customisation data - viz. the user topics and SitePreferences. Using it for this purpose actually makes the name "Main" more sensible - it's got a broader role than just users. It will require careful (and easy to find) documentation, detailing how to customise it.

-- CrawfordCurrie - 07 Oct 2009

Yes, I plan on adding a HowToCustomiseDefaultTopics in Support.SupplementalDocuments for this, as well as the shipped documentation.

-- AndrewJones - 07 Oct 2009

Imho, adding a default to include is the wrong approach. VIEW_TEMPLATEs is better and much more flexible. Besides, it does not need any changes to the core. Why has this initial idea been abandoned?

I'd create

• System.WebAtomViewTemplate
• System.WebChangesViewTemplate
• WebIndex delete this one
• System.WebRssViewTemplate
• System.WebSearchViewTemplate
• WebSearchAdvanced fold it into WebSearchViewTemplate
• System.WebTopicListViewTemplate
• System.UserRegistrationViewTemplate
• System.WebCreateNewWeb (new)
• System.WebCreateNewWebViewTemplate
• System.WikiUserViewTemplate
• System.WikiGroupsViewTemplate

and then add VIEW_TEMPLATE preference settings to the respective topics:

• WebAtom
• WebChanges
• WebRss
• WebSearch
• WebTopicList
• UserRegistration
• WebCreateNewWeb
• WikiUsers
• WikiGroups

The view templates are constructed along the lines of WebCreateNewTopic which thus allows to override specific parts of the form easily. The INCLUDE approach won't allow me to reuse any bits in the original view template.

Moreover, changing the skin would allow me to switch all view templates automatically when the skin comes with custom dialogs. Using the INCLUDE approach you'd end up to select variations one by one. Error prone.

-- MichaelDaum - 07 Oct 2009

I find the template approach way too geeky. For a beginner that has just installed Foswiki, understanding our template system is near impossible. Even I still struggle with it.

It is much easier to understand a normal topic. You copy the content of the default and make your own. It is easy. And it should be easy.

If we go the template path people will continue to change the distributed topics instead and we will have won nothing.

We have to be able to put ourselves in the beginners shoes.

Besides, the INCLUDE enhancement is nice and very useful for many other things. It is not a hack for specific purpose. It has a lot of good use and easy to understand and document.

The example of WebCreateNewTopic is the best example of why this is a user-UN-friendly approach. If you look at WebCreateNewTopic your first question is: "How do I edit this topic?". And once you realize that you have to put ?raw=debug on the URL you see a setting of a VIEW_TEMPLATE to WebCreateNewTopic. Duh!

The beginner is now expected to know that he must go to the system web and look up WebCreateNewTopicTemplate. And the content of this to a beginner is utter garbage. No. That is not the way I want to help a new admin to tailor what is simple things in Foswiki. You should be able to create and edit normal topics in normal TML whereever possible.

The good thing with the INCLUDE is that the user can easily figure out what to do because the INCLUDE is self explaining. And because it is an INCLUDE we can write a small instruction of how to create a tailored version in the default topic outside START/STOPINCLUDE tags.

-- KennethLavrsen - 08 Oct 2009

Kenneth is right: Although the template way is more flexible, its far too complex to grasp for new users. I prefer the topic-suffix way.

-- OliverKrueger - 08 Oct 2009

I'd like to see a pros/cons breakdown of the two approaches. On the face of it, the INCLUDE approach seems a lot simpler, but that may be because I just don't fully understand Michael's proposal.

-- CrawfordCurrie - 08 Oct 2009

Pros and Cons

INCLUDE Approach

• Pros:
  • Easy for new users
  • INCLUDE is well used and well documented
• Cons:
  • If a skin/extension wants to make changes to more than one of those topics, I will need to make changes to all of the topics, which is tedious and error prone
  • Requires a code change
    • Arguably, thats a good change to make anyway

Templates Approach

• Pros:
  • Templates are much more flexible than includes, can drastically change the layout of a page
  • Skins/extensions would then be able to ship their own versions of topics
    • for example, KinoSearchContrib could ship a new WebSearch called KinoSkinWebSearchTemplate and the user can append kino to their skin path to use it. Would be even better for skins.
  • Consistent with WebTopicNewCreate
• Cons:
  • Templates are more complicated
    • need to learn and understand the mark-up (TMPL:P, etc)

---

Started a pros and cons above, please expand. Once they are gathered we can refactor the topic.

With regards to templates being too complicated, if we can improve the docs I am sure we can lower that barrier.

After writing the pros and cons, I think I am going to side with the Templates approach (as long as I/we can get the documentation right).

-- AndrewJones - 08 Oct 2009

I am a bit confused. There are two or three things that we better don't mix up.

Normal uses just use these admin tools, like clicking on "new topic". We should always strive to hide complexity from normal i.e. unexperienced users. We need to hide things they don't understand. That's because they don't want to understand it. They want to do their job, like creating documentation.

I think everybody agrees.

So even an %INCLUDE as much as it is self-explanatory is too much. When a normal users opens up a topic to edit it and sees an %INCLUDE he will definitely be confused because he does not know it. At best he will become curious and take the named topic name and look it up somehow. Note, it is not clickable. He will have to close the editor and some jump feature to go to the topic named in the %INCLUDE - a major disruption of his workflow.

The best means to hide this application-ness from users is VIEW_TEMPLATEs.

A normal i.e totally unexperienced users does not care if the tool for his job is build using INCLUDE or VIEW_TEMPLATE. That's because it is not relevant for the job he wants to do with this tool.

Therefore, the "too complex for new user" argument is a bit off. Only iff this user manages to find the VIEW_TEMPLATE he will kind of get shocked. But why should he do that?

Only when he wants to understand where this tool he sees is coming from will he start to look up its implementation. This, however, is not normal for normal users.

@Crawford: the most simple way to convert WebSearch to the described approach

1. create a topic System.WebSearchViewTemplate
2. add %TMPL:INCLUDE{"view"}%
3. add %TMPL:DEF{"content"}}% <copy of former WebSearch content> %TMPL:END%
4. erase all content from WebSearch
5. edit the topic preferences of WebSearch and add Set VIEW_TEMPLATE = System.WebSearchView

This definitely works though it is not a finished job. as the content of WebSearch needs to be modularized into several TMPL:DEF bits to be easily reusable. For example lets create a TMPL:DEF{"input"}, TMPL:DEF{"options"}, whatever, and move the html bits from their original location into these macros. Put a TMPL:P{"input"}, TMPL:P{"options"}, at the place where to expand the macro. Most probably creating the user interface for WebSearch needs a usability overhaul at the same time but thats a different story.

Creating a custom WebSearchViewTemplate goes like this:

1. create a topic System.KinoWebSearchViewTemplate
2. add %TMPL:INCLUDE{"WebSearchView"}%
3. add %TMPL:DEF{"options"}% <some kinosearch relevant options> %TMPL:END% to override the standard definition

Proceed along the same line for other kino-related templates. Then add kino to the SKIN to switch to the customized versions of all dialogs. Or let a plugin do it automatically as soon as it gets installed, e.g. KinoSearchPlugin prepends the kino id to the SKIN preference. So you unzip the plugin, enable it, and all is kino-fied already.

-- MichaelDaum - 08 Oct 2009

WebSearch is an interesting example, as that really needs a way to not just replace parts, but to extend them. For example, when installing TagsPlugin, I want it to add a tag search checkbox, and some tag relevant options. And then, when I install ExternalDocumentSearchContrib, it would add another search type, and so on....

As Michael shows though, before 1.1/2.0 we need to continue and complete this discussion, so we can decide on which one mechanism we think is best.

wrt VIEW_TEMPLATE / skin solution - I guess the missing feature is to implement recursive TMPL:DEF's

ie

%TMPL:DEF{"options"}% <some kinosearch relevant options> %TMPL:P{"options"}% %TMPL:END%

whereas if we were to use INCLUDEs and sectional INCLUDEs, the equivalent already exists, but chaining becomes pretty darned complicated.

I'd shy away from a plugin adding into the SKIN path - as a Plugin could get the ordering wrong, or worse, might add kino into the path, when the end user has written a myownkino.

ok, so as usual, i favour the SKIN path approach...

-- SvenDowideit - 10 Oct 2009

+1 to recursive TMPL:DEFs

-- MichaelDaum - 10 Oct 2009

When I nodded to Kenneth, I had the "new" admins in mind, not the end-user. But maybe I'm too protective towards padawans.

-- OliverKrueger - 10 Oct 2009

I think we may not be talking about the same things.

I can see the debate is targetting how to write your own skin more or less.

The issue are those tailored topics where the topics in question is simple TML and where the tailoring is simple TML changes.

This is where the TMPL:DEF is a geeky overkill. To be able to change such a template you need to be pretty well familiar with the templates that ships with Foswiki.

In practical life - you have to be a skin author to fully have the overview required to both

• Know how to build a template that modifies e.g. the normal view template
• Know how to do it so it still works after the next upgrade.

When we take topics like WebLeftBar, WebTopBar, UserRegistration, etc - you tailor these today by editing simple TML.

Why should something that is today defined by a simple list of bullets suddenly require detailed knowledge about the Foswiki template system?? No way. It is far far far to geeky. Even I - who has been working on this project for years - am not able to modify the most simple things in the templates. I spent 5 hours last time I needed to add a simple thing to a view template. I am an expert. I spend half a working day on a simple thing. The Foswiki template system is very complicated to understand. Just a simple pattern skin view template is spread over many files. Trying to understand how to modify it requires deep inside knowledge.

It is easy for Michael that wrote his own skin. And for Sven who also wrote a skin. But it is not at all easy for a normal person installing Foswiki and wanting to just tailor a couple of the fields in the user registration form.

I actually see changing the WebSearch form as a skin modification. That can be done the template way. WebSearch is not a normal topic and people wanting to modify this are already deep into hardcore UI design.

But for example tailoring the WebRss to meet a specific requirement like including some field from a form in the search results should be a matter of editing a simple topic. And here a conditional INCLUDE could be a simple and easy to understand way.

-- KennethLavrsen - 12 Oct 2009

Seems you won't turn in. Let's settle this dispute by extending the AutoViewTemplatePlugin the way I described in AddAutoViewTemplatePlugintoFoswiki. That way arbitrary topics can get a VIEW_TEMPLATE overlay that leverages the flexibility as I outlined above while people who use INCLUDEs can still do so.

-- MichaelDaum - 12 Oct 2009

That does not solve the problem with e.g. UserRegistration.

The reason people tailor this topic is that they match its fields to the fields they tailor in the UserForm.

We have a solution today for UserForm. I can make my own in Main web and it overrules the one in System.

We need a similar mechanism for UserRegistration. Today I avoid including this in the upgrade package but recently it was updated so these improvements will not be present in an upgrade package next time we release 1.1.0.

It would be nice if we changed this to UserRegistrationDefault and allowed people to tailor their own.

And in this the INCLUDE feature could do the job. The tmpl method would suck and be way too geeky. Arthur did a good job with WebLeftBar etc. And that is done with some IFs in the skin templates. It is not equally easy to do with UserRegistration.

There are more ways to do this. But what ever we do it has to be something that a person can do in his first 10 hours of experience with Foswiki. Ie a new admin who it setting things up for the first time.

-- KennethLavrsen - 18 Oct 2009

Maybe you are right. You don't get it so everybody else won't get it either.

UserRegistration is really a bad wiki app spotted. E.g. too many questions for an innocent person to answer just to get an account. It makes some wild assumptions wrt allowing login names and using passwords that don't make sense to me.

In up to all foswiki/twiki installations I have seen, admins did not stay with the standard UserForm, UserRegistration and NewUserTemplate. These have next to always been kicked out and replaced with something less cluttered making sense to the client needs.

-- MichaelDaum - 19 Oct 2009

Im not sure where this proposal is at now.

I would disagree that templates are too geeky and difficult, and suggest that they are just badly documented. There isn't much at System.TemplateTopics. With examples I think that it would be quite easy for new admins to customise a template topic.

UserRegistration is a special case. Im not sure about the best approach for this.

-- AndrewJones - 21 Oct 2009

Yes right.

-- MichaelDaum - 21 Oct 2009

Proposal for RulebasedViewTemplates. Includes implementation.

-- MichaelDaum - 03 Nov 2009

I will park this proposal, as unfortunately there was no agreement.

Its a shame, because IMO the System web is one of the most annoying/broken parts of Foswiki, and this started off as a simple proposal to make customising some of the topics a lot easier.

If people revisit this proposal and can work out how to get past the objections then I might still commit to doing the work, but it does not look like a consensus will be reached.

-- AndrewJones - 17 Feb 2010

Follow up discussion

Inserted: I posted 2 (I thought simple) solutions, but those do not work out: solution 1 cannot work, solution 2 is not better than the proposed INCLUDE extension. I have now reverted the proposal to the original state.

I have kept some of the follow up discussion below. -- ArthurClemens - 02 Mar 2010

I don't understand the requirement that it has got to be simple. This is for admins, not users, so we can assume some knowledge of Foswiki or at least someone willing to make the investment to learn about the templates. I also don't think that templates are that difficult to use. The docs at TemplateTopics can be improved, and nearly all administrators will already have customized the skin so will already be familiar with the templating syntax.

• Not all admins are changing the skin. A simple Google search will point you to lots of uncustomized sites (except for a logo). The templating system (as any) is complex for beginning users, especially if they are not motivated to learn any of it. And it scares the hell out of them. -- AC

-- AndrewJones - 25 Feb 2010

A quick Google did indeed prove the point. I would accept an INCLUDE based approach, which is still a huge improvement.

-- AndrewJones - 26 Feb 2010

I like the proposed default parameter for INCLUDE. Very useful. Even more if default was a list of topics, a kind of search path.

However, I still think that a TMPL-based approach would work out better. The major differences is that, while you specify a single default value inside the INCLUDE call explicitly and a priori, the TMPL mechanism decides on the current view following the search path. Another advantage of the TMPL mechanism is that definitions can be redefined. You can't do that for named sections.

These two advantages of TMPL over INCLUDE make it a superior tool for upgrade-safe topics.

Using either a set of topics and default topics with lots of sectional includes or view_templates is similar in complexity. The TMPL mechanism is a proven technology, whereas default for INCLUDE isn't even implemented.

That's why I still have concerns.

There is a middle ground of using VIEW_TEMPLATES that then INCLUDE topic functions to render parts of a page, but I won't recommend that here.

-- MichaelDaum - 02 Mar 2010

We have recorded your concern. It is now time to put forward the original SIMPLE normal people's solution and get a community vote. We are not going to agree by continuing with the same arguments that lead to disagreement.

But I need Arthur to refine the description above so it is clear to us all. And then we have to take a vote to get this one accepted or rejected

-- KennethLavrsen - 02 Mar 2010

Of course there is disagreement. Arthur has emailed me and asked to follow up on the latest arguments. And that's what I did. I don't have other arguments and I feel they are still valid.

-- MichaelDaum - 02 Mar 2010

If we are going to decide this by vote, can we vote for the 3 options? i.e:

• INCLUDE based approach
• TMPL based approach
• Neither (reject both)

If so, I can commit to implementing the TMPL based approach, based on the original proposal at http://foswiki.org/Tasks/Item1935?rev=1 (which can be copied in here, if this will be an option in the vote).

-- AndrewJones - 02 Mar 2010

Sorry guys but I messed up. My proposal does not work. And the alternative proposal is not better than the INCLUDE extension.

I think now that the default parameter to INCLUDE is the best middle ground between functionality and usability. With a naming scheme using DefaultXxx syntax.

-- ArthurClemens - 02 Mar 2010

Michael and Andrew

Please try. Just try and understand the situation that we are trying to resolve and please try and put yourself inside the person I will describe now.

Let us call this person Jens Smith.

Jens needs a web based tool for some department function. He want to share some knowledge. He has some static HTML pages done in some Wysiwyg tool but never updated. He has some ideas of some vacation lists, some work flow applications. He does not really know what he wants but then he reads about wikis and he knows wikipedia. After having compared different wikis he decides that Foswiki is the tool to try because it claims to have features for making simple applications.

Jens know little about programming. He has programmed a few Basic macros in Excel and some 5 lines scripts in DOS. But he has never written a single line of perl. But he is not afraid of trying something new and with all detailed installation documentation he decides to make a try.

He asks a friend to help him install a basic Linux. He chooses Ubuntu because it seems to be very newbie friendly and it is popular.

So he follows the installation documentation for Foswiki and he is good at following a step by step approach. He has a few problems with some access rights but after a day or two he has an out of box Foswiki running.

He follows the tutorial and after a few more days he can make a table, some bold text and headlines. Nothing fancy.

He is now ready to tailor it a little so it gets some looks of being part of the company.

He finds out how to put his company logo in the top bar. But all the internal webpages in the company has a different background colour so he finds the documentation that advice him to not touch the WebTopBarExample but instead make a copy of it called WebTopBar. He has no clue what the all the stuff in WebTopBar is but he is able to tweak it a little so he gets a different colour, and adding some link to the corporate portal.

His next step is to tailor the user topic form a little. He does not need the country. But he would like a field for the room number in the building of the person.

Again Jens is a smart guy and reads the documentation and reads that editing UserForm in System web is wrong because it gets overwritten when you upgrade Foswiki. Instead he follows the documentation and creates a copy in Main web and he then removes and adds the fields he wants changed.

He also copies the NewUserTemplate to Main web and replaces the UserForm with the one from Main and he does some other simple changes. After all it is just plain text and bullets. Easy for a newbie to understand and change.

Now the trouble starts.

He wants to change the UserRegistration topic to reflect his changes to the user form. But there is no documentation that tells how to make a copy and tailor the copy. So he tailor the one in System web because this seems to be the only way. And this is indeed the case.

Next time he upgrades to Foswiki 1.1 his UserRegistration gets overwritten and he is upset.

Same thing happens with some minor tiny little changes he does to 2 of the other Web* topics.

Let us stop the story here.

Jens is the typical Foswiki admin. The typical admin knows no perl, no bash, may be a Windows user installing on Windows, has never used Foswiki before.

The typical Jens knows nothing about view templates. He knows nothing about template paths. His head explodes when he sees TMPL:DEF and TMPL:P and has no clue what that is.

Truth is that for many of us that are contributing to Foswiki we still mess around and do not quite understand what is going on with this very strange template syntax which is very different from the TML a normal user knows.

The very flexiple but very very complicated template path which depends on Skin settings, can find templates in template directories as well as webs, assumes that topic templates are named without the trailing *Template etc etc. It is a highly complex system meant for building skins.

It is totally absurd to think that any newbie admin can take advantage of this.

Just today I saw on IRC...

[12:08] <CDot> hmmm, where does new topic creation get it's view template from?
[12:08] <CDot> as in, type in the URL of a non-existant topic, where is that page templated?

This is Crawford being confused!! Imagine a newbie admin.

Whatever technical solution we find for the problem "Allow users to customise System topics (such as WebSearch) in an upgrade safe way" - make the solution with Jens Smith and all our new admins in mind. And remember that technical smart solutions with huge flexibility is rarely what people need.

They need to make small tweaks to things that are already there. We need to enable them to do that "..in an upgrade sae way".

I hope this triggered some 2nd thoughts. AND some new SIMPLE ideas.

-- Main.Kenneth "Tolstoj" Lavrsen - 02 Mar 2010

Sorry, Kenneth, I don't have time to read such lengthy comments. I am remoing myself from ConcernRaisedBy field to let this one go wherever it goes. Maybe I am wrong and you are right. We will see.

As there's a good basis for easy to maintain upgrade-safe wiki apps available elsewher, I don't see the point of endless discussions here. Good luck.

-- MichaelDaum - 03 Mar 2010

Kenneth, I agree and can understand the need for someone like Jens to quickly and easily change topics such as UserRegistration and WebTopBar, after using Foswiki for just a few days.

However, the following are the topics I originally wanted to be customisable using TMPLs:

• WebAtom
• WebChanges
• WebIndex (or delete it all together)
• WebRss
• WebSearchAdvanced
• WebSearch

Users like Jens would not want to change these topics. The defaults are fine for him. But experienced admins might want to, and plugins also might want to provide different templates for these topics (in particular the search topics). For these topics I still feel that the TMPL based solution is the best, as it does not need to be easy for newbies and the flexibility of TMPLs make it more powerful for experienced admins and (more likely) plugins.

The problem with this proposal is that it grew to cover all default system topics, possibly because of the name of the proposal. I want to focus just on these for now. UserRegistration does need a solution, which should be discussed in MakeUserRegistrationCustomizable.

-- AndrewJones - 03 Mar 2010

For WebAtom and WebRss a common mini-tailoring is to add some excluded topics to the search because people do not want some of the more noisy topics to appear in the RSS feed. E.g. I often exclude the WebNotify topic and some other household topics. In our Task web the autoupdated HallOfFame became a pain in the butt to see every 2nd hour in the RSS feed.

I have also done some tailoring once where I modified the WebRss to show some specific form fields in a web dedicated to bug reports. But that is a more advanced type of tailoring and not a Jens Smith thing.

So those topics do get very simple tailorings where a template solution would be an overkill.

I agree that the 3 WebSearch / WebSearchAdvanced / WebChanges topics could be template based. I doubt many need to tailor those. Or they will create a new Search topic for specific use and link to those via WebLeftBar. I can agree that those are template tailored. WebIndex - I use it often when I cannot remember what a topic is called. But I never imagined tailoring it. So that could also be template based. But please do not delete it.

So in summary - I would like the simple solution for WebRss, WebAtom and UserRegistration. And I am OK with the template solution for the WebSearch / WebSearchAdvanced / WebChanges.

Does that make sense?

-- KennethLavrsen - 03 Mar 2010

I would like to separate this proposal out a little please.

Would we all agree to adding AddDefaultTopicParameterToINCLUDE so that can proceed? (I spend too much effort reproducing this functionality using a IF's etc..

additionally, I would like this proposal to cover the leftbar case - where we ship a default left bar that is used for all webs, which can be over-ridden by the site's admin, then allow that topic to be over-ridden using a web based one, and subsequently allow users to also over-ride that.

which supports Michael's point that INCLUDE needs a path of topics to test...

INCLUDE{"%!BASEWEB%.%!WIKINAME%leftbar" default="%!MAINWEB%.%!WIKINAME%leftbar,%!MAINWEB%.%!WIKINAME%leftbar%!BASEWEB%.leftbar, %!MAINWEB%.defaultleftbar, %!SYSTEMWEB%.defaultleftbar"}

but that suggests that maybe we should just make the _DEFAULT topic parameter a list that can be tested... (i'm ignoring my other use case, where i have different topics with different sections, cos that gets complicated quickly, and was probably a bad idea

-- SvenDowideit - 04 Mar 2010

Kenneth, yes that makes sense. I will leave WebAtom and WebRss for now, and just apply the template solution to WebSearch, WebSearchAdvanced, WebChanges and WebIndex (so WebIndex won't be deleted, but can be tailored). That is all this proposal will cover, and it covers the topics I most cared about (i.e. search), so I have switched to UnderInvestigation and added todays date.

Sven, I like the INCLUDE ideas, and they can be done in a different proposal. Changes to WebLeftBar can also be done in a different proposal.

Later I will re-factor the proposal to make it more clear.

-- AndrewJones - 04 Mar 2010

With Andrew's wrap up - my reason for concern can be resolved.

But I cannot really raise my concern because the original proposal still says something very different from what consensus is about.

We need TWO things now.

• Andrew please rewrite the proposal to what we have consensus on with respect to the topics with which we will use the template method

• The rest of you - do we have consensus on implementing %INCLUDE{"Webname.Topicname" default="Webname.Defaulttopicname"}% to be used as a general feature incl tailoring of the ones where we do not want to use the template method because they are newbie admin scoped.
• If we accept the INCLUDE extension - can we agree on the default being a list? if so please write this in the proposal. If we disagree I will split the INCLUDE out in a separate proposal and give it its own proposal spin.
  • I've already made AddDefaultTopicParameterToINCLUDE though I also wonder if we can just drop the new default parameter, and just allow the topic specifier to be a list of topics..

-- KennethLavrsen - 09 Mar 2010

I don't really think we want to use 2 methods at the same time, and while i prefer the INCLUDE method, and will need to craft an alternative to find out if its possible, looking forward to seeing the template proposal in full.

-- SvenDowideit - 10 Mar 2010

Sorry it took a week to re-factor this proposal. Could you please take a look and see if your concerns are resolved?

-- AndrewJones - 16 Mar 2010

Looks fine to me.

-- ArthurClemens - 16 Mar 2010

Ok with me too

-- KennethLavrsen - 17 Mar 2010

SvenDowideit, do you still have a concern? Or do we have a ConsensusReached?

-- AndrewJones - 22 Mar 2010

Yes, I am still concerned.

I suspect 'This would not be a possible using AddDefaultTopicParameterToINCLUDE. ' is simply incorrect and the current proposal is basically premised on the idea that 'this is the only way to doit'

Conversly, as someone that has always advocated that we should be using skin path more..... i'm a little conflicted

I definitely am concerned that MakeUserRegistrationCustomizable and this proposal, WebLeftBars, TopBars and the like all need to be implemented using the same technique - else customisations will become more confusing for those that we're supposedly helping.

-- SvenDowideit - 25 Mar 2010

After a night of sleep, I have decided that this is not ready for community vote. We are still getting ideas to new solutions to this problem. I add my concern again.

I do not agree with Sven that all the different topics need to be customized using the same technique.

But I am also deeply concerned about using skin paths for doing something that makes other uses of skin path - like real tailorings of the skin more complicated.

If someone want to change the general look of the PatternSkin by setting a skin path and making a number of templates for specific changes, having to consider also tailorings just makes it even more complicated.

Another issue is that these template solutions will lock down the distributed topics so we cannot easily changes them. The template name scheme will become a sort of API in themselves that we cannot change in future without giving trouble for some people. If they use a TMPL:P{"somename"} in their tailored template and we change the somename nothing works. I see a nightmare future.

And when I then think about the list:

• WebAtom
• WebChanges
• WebIndex
• WebRss
• WebSearchAdvanced
• WebSearch

then I come to think about a new simple solution

These topics can be tailored by copying them to a new topic and simply have WebLeftBar and WebHome to point to the tailored versions. That is where these topics are used.

Why make things so complicated when there is a simple solution?

We should just define a list of topic Macros like

• Set WEBATOMTOPIC = WebAtom
• Set WEBCHANGESTOPIC = WebChanges
• Set WEBINDEXTOPIC = WebIndex
• Set WEBRSSTOPIC = WebRss
• Set WEBSEARCHADVANCEDTOPIC = WebSearchAdvanced
• Set WEBSEARCHTOPIC = WebSearch

in DefaultPreferences and use these Macros in WebLeftBarExample and WebHome's.

Sometimes we try to create very complex solutions to what is actually a simple problem.

I have also suggested this same solution in MakeUserRegistrationCustomizable which actually suits Sven's wish for same solution for all tailorable topics.

I am also sure that such a macro solution is must faster than e.g INCLUDEs that have to fail finding a topic before going for the next in the list. Or template paths that have to try finding the right template in a long path.

Last, the macro solution is backwards compatible. As long as we define the topic names as they are today, upgraders will not see a difference.

We have not considered all options yet. Let us continue the discussion. It is not a killer feature to provide a solution for topics that people hardly ever tailor anyway.

-- KennethLavrsen - 25 Mar 2010

Yes, it requires a kind of switch board to select which alternative implementation to use. But what about reusability? Imagine, you would like to make a simple tweak only. You'd have to copy & paste all of the original wiki application for that and end up in lots of redundancy. This needs a more fine grained way to address parts inside the application, either by calling named sections or by redefining tmpl macros.

-- MichaelDaum - 25 Mar 2010

'This would not be a possible using AddDefaultTopicParameterToINCLUDE. ' is, IMO, correct. Using WebTopicList as an example, it might have the following INCLUDE: %INCLUDE{"WebTopicList, WebTopicListDefault"}. Now a plugin ships its own WebTopicList, and the admin has an extra step to copy and paste MyPluginWebTopicList to WebTopicList. They have to do this manually, and its not a good solution.

Using Marcos for the links would work for new wikis, but any existing webs will still have links to the old page. We have >60 webs, I don't want to change all the links. Also people may have bookmarked the topic (in particular the search topics). We had a similar problem on foswiki.org, where we had to update our bookmarks from SiteChanges to FoswikiSiteChanges because the upgrade overwrote the changes. Not a good advert.

Perhaps a compromise would be to have an INCLUDE like this: %INCLUDE{"%WEBTOPICLISTTOPIC%, WebTopicListDefault"}. It doesn't allow resusability, but at least it provides some solution, because at the moment there is no good solution.

I do agree that it would be best if we had the same solution for all system topics. However, I think UserRegistration is a special case, as it needs to be easy to customise when setting up Foswiki for the first time. But for Web* topics, it should be the same. At the moment, this is the only solution currently in use, as it is used for WebCreateNewTopic. So if we chose a different solution and updated WebCreateNewTopic, it would actually break backwards compatibility.

I would really like a solution for 1.1, as this is one of my major annoyances with Foswiki.

-- AndrewJones - 25 Mar 2010

Talking on IRC a long time ago with SvenDowideit putting Web* under a common WebTools topic parent one day, the idea of phantom topics was mentioned.

At the time I sort of imagined plugin code that could arbitrarily define its own mapping for where it really pulled a topic from when you tried to access a "missing" topic of a special name.

The idea is:

• Allow a new "empty" web to simply contain a WebPreferences.txt. WebSearch, WebIndex etc. would all be special missing topics whose names are registered to be found at an alternate location: in some dedicated _phantomweb, or series of them - defined with a SKIN-path esque type variable

Eg:

• Set PHANTOMWEBS = _MyPhantomWeb, _KinoSearchPhantomWeb, _defaultphantomweb

This doesn't cater for how we selectively override parts of a search, but we could use some mix of ADDTOZONE and INCLUDE paths to do it.

_MyPhantomWeb.WebSearch

%ADDTOZONE{"searchcontrols" text="<input type='checkbox' name='fooOption' value='fooValue'/>"}%

_KinoSearchPhantomWeb.WebSearch

%ADDTOZONE{"searchcontrols" text="<input type='checkbox' name='kinoOption' value='kinoValue'/>"}%

_defaultphantomweb.WebSearch

%ADDTOZONE{"searchcontrols" text="<input type='checkbox' name='mainOptions' value='mainValue'/>"}%

%STARTSECTION{"searchcontrols"}%
%MAKETEXT{"Search options: "}% %RENDERZONE{"searchcontrols"}%
%ENDSECTION{"searchcontrols"}%

%STARTINCLUDE%
%INCLUDE{section="searchheader"}%
%INCLUDE{section="searchhelp"}%
%INCLUDE{section="searchcontrols"}%
%INCLUDE{section="searchinput"}%
%INCLUDE{section="searchfooter"}%
%STOPINCLUDE%

Maybe rather than sections it should be TMPL:DEFs and INCLUDEs should be TMPL:P, as these are already "Skin path" aware, but I would rather see better accessibility of this feature for users who are already familiar with sections/includes.

-- PaulHarvey - 25 Mar 2010

given, AddDefaultTopicParameterToINCLUDE

we get the possibility to have %!INCLUDE{%!WEBATOMTOPIC%}%, which can then be set on the SitePreferences topic (using a single topic, or even a list of them, or, using a SEARCH to automatically generate the list) - with the same, but more flexible approach as prefixing the skin path.

just that makes This would not be a possible using AddDefaultTopicParameterToINCLUDE incorrect.

From that simple beginning, we then can develop the idea further using the methods I've been doing in WidgetsSkin and FoswikiApplicationsContrib, or the rather interesting use of webs that Paul enumerates.

Um, Paul, I think you might be wrong wrt This doesn't cater for how we selectively override parts of a search - as AddDefaultTopicParameterToINCLUDE means that each of the sectional includes could be written as:

_defaultphantomweb.WebSearch

%STARTSECTION{"searchcontrols"}%
%MAKETEXT{"Search options: "}% %RENDERZONE{"searchcontrols"}%
%ENDSECTION{"searchcontrols"}%

%STARTINCLUDE%
%INCLUDE{"%WEBSEARCHTOPICLIST%" section="searchheader"}%
%INCLUDE{"%WEBSEARCHTOPICLIST%" section="searchhelp"}%
%INCLUDE{"%WEBSEARCHTOPICLIST%" section="searchcontrols"}%
%INCLUDE{"%WEBSEARCHTOPICLIST%" section="searchinput"}%
%INCLUDE{"%WEBSEARCHTOPICLIST%" section="searchfooter"}%
%STOPINCLUDE%
   * Set WEBSEARCHTOPICLIST = %FOREACH{"%PHANTOMWEBS%" format="$item.WebSearch"}%

however, you are right that we're missing a way to add elements using this method.

This discussion gives me hope that some day soon we will be able to deprecate the TMPL system, and use sectional includes to achieve the same result. (not to say we want to, more that we're succeeding in making the user facing syntax expressive enough to have enough flexibility)

Andrew, the key concern I have is that we should not force our new admin to have to figure out which of the multiple techniques are to be used to over-ride something. IMO we need a designed approach, which thus requires more time and thought.

Whenever there is a proposal that insists this is the 'only' solution, and someone else figures out an alternative, We can't afford to rush.

-- SvenDowideit - 25 Mar 2010

Thanks for your discussion. I do also feel its important that we should use one technique, and right decision is made.

I am warming to the %INCLUDE{%WEBATOMTOPIC%}% idea. Although I don't see it gives us any benefits over the templates approach, but it is arguably easier for new admins (although my argument was always that this is not a feature for new admins, but admins who have been using Foswiki for months). I also like the idea of using sections.

Im not sure about the phantom web idea. Seems to be more complex and I don't see many benefits.

One problem we have at the moment is that WebTopicList has been implemented using templates. If we decide to use INCLUDE's, that will need to change so it can be overwritten using the same technique. The change would not be backward compatible.

-- AndrewJones - 26 Mar 2010

It is a loss that we could not agree before the code freeze for 1.1. Now we only have made the user registration page update safe.

-- ArthurClemens - 14 Apr 2010

With RecursiveTMPLDefinitions implemented, we need to have a fresh look at this proposal.

-- ArthurClemens - 03 Jan 2011

Removed the PlannedFor 1.1, long since passed. This doesn't appear to be going anywhere. Changed to Parked

-- GeorgeClark - 09 Feb 2015

Commenting is disabled while not logged in