Feature Proposal: Implement dynamic views for topic lists

Motivation

We have a way to view and edit individual topics using the TEMPLATEs, but what we are missing is a pluggable way to do the same when viewing a list of such topics.

http://foswiki.org/System/SiteChanges?raw=debug shows a poorman's, hardcoded version of this on a per web basis..

Description and Documentation

In FoswikiApplicationsContrib, I have been defining the individual view and list view*edit displays in topic sections - now that we have AutoViewTemplatePlugin, we can add listviewtemplate and listedittemplate, LISTVIEW_TEMPLATE and LISTEDIT_TEMPLATE, and amend the default SEARCH output formats to leverage them.

The proposal dated 25 Jun 2011 differs from this earlier idea. -- AC

That way, the Mail Notifications for the Tasks and Support web can emphasize the changes better, and table based applications can even have inline editing.

Examples

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

-- Contributors: SvenDowideit - 05 Nov 2009, ArthurClemens - 25 Jun 2011

Discussion

I'm not clear what the proposal actually is ..... are you proposing adding parameters to %SEARCH? Or something else?

-- CrawfordCurrie - 05 Nov 2009

Let me outline the mechanisms I've been experimenting with in DBCachePlugin and WikiWorkbenchContrib. There's an extended semantics (compared to INCLUDE) that derives the TopicFunction to render a list item based on the TopicType property of an attached form. This looks like this (percnt and dollar removed for clearness):

SEARCH{

  format= DBCALL{$web.$topic->RenderListItem}

}

This will analyze the topics found by SEARCH and checks if there is a matching implementation of RenderListItem for it.

So given one of the found topics has the property

TopicType = MovieTopic, CategorizedTopic, TaggedTopic

It then checks for

1. Applications.MovieApp.MovieTopicRenderListItem
2. Applications.ClassificationApp.CategorizedTopicRenderListItem
3. Applications.ClassificationApp.TaggedTopicRenderListItem
4. Applications.RenderListItem

in the given order. (1), (2) and (3) are found as these TopicTypes are known to be implemented in the given webs (another story). (4) is a standard fallback. If one of the TopicFunctions exists it is called using a parametrized include with a mandatory parameter OBJECT holding the current $web.$topic setting. So any of the RenderListItem implementations can then generate a snippet to represent this topic in the hit set. E.g.

<div class='movieTopicListItem foswikiListItem'>

[[%OBJECT%][FORMFIELD{"TopicTitle" topic="%OBJECT%"}]]

Rating: FORMFIELD{"Rating" topic="%OBJECT%"}

IMAGE{
  "FORMFIELD{"cover" topic="%OBJECT%"}"
}
</div>

Note, that this approach allows to reuse and override on a per topic base how the items in the hit set are rendered.

-- MichaelDaum - 05 Nov 2009

It seems to be a generic problem among developers that you always forget the date of commitment. Maybe we need to change the application that creates proposals to force this. I cannot make it mandatory field because it should only be set if CommittedDeveloper is set.

I added todays date. And added concern by Crawford and myself because it is a proposal without spec. It states a problem and a desired end result but not how the feature is to be made and used. Ie. no syntax of use. And this is the only reason for concern from me. The problem and the desire to support it I support. I just don't want a 14-day acceptance of something that I do not know how is going to look like for the end users.

-- KennethLavrsen - 05 Nov 2009

I've been thinking about how to implement this, and have started to think that yes, it may well require little more than documentation to guide DataFormApplication Developers and the $include() format specifier that I have in another feature request.

eg,

In the DataForm definition topic, optionally add a SECTION{listviewtemplate} and SECTION{listedittemplate} which if defined will be used to render the resultant topic's line in WebChanges, WebSearch and any other format based result..

and then we amend WebChanges etc to SEARCH{format="$listviewtemplate"} (or something) where $listviewtemplate is equivalent to INCLUDE{"FormDefinition" section="listviewtemplate" default="NormalTopic"}

the $listedittemplate then will allow inline editing of topics directly from a WebSearch which would make changing the status of a task soooo much easier.

basically, Micha and I have done the same thing - Me using existing core features, and his using a more magical plugin.

I'll update the User Management demo application to match the above, and then show it as a further example.

-- SvenDowideit - 22 Jan 2010

Let me refactor what I outlined above a bit so that it doesn't rely on TopicType.

Given a topic MovieTopic has got a form attached to it MovieForm.

Then

%INCLUDE{"MovieTopic->ListView"  ...}%

will be processed as follows:

1. look up the form definition of MovieTopic ... which is MovieForm
2. strip off any occasional ...Form
3. append ListView
4. execute an %INCLUDE{"MovieListView" ...}%

This will look like this in a %SEARCH

%SEARCH{

   format="$percntINCLUDE{$topic->ListView}$percnt"

}%

Note, that the "object call" is not restricted to formatting list items.

We might think of other strategies to derive the target template to be loaded similar to AutoTemplatePlugin, like a topic-name based one which will process the topic name instead of the form name when a topic doen't have a DataForm attached to it.

-- MichaelDaum - 22 Jan 2010

too late for me to look into in the next release timeframe - parking.

-- SvenDowideit - 06 Mar 2010

Sven was right that this proposal is already covering what I tried to describe in TemplatingSearchResultSummaries.

There are now 3 proposals how to implement this:

1. Using a template mechanism similar to AutoViewTemplatePlugin: a topic with form ProjectForm will be displayed in search results using markup in template ProjectListViewTemplate
2. Using an include mechanism that gets the included section from the form topic (this is also used in AutoViewTemplatePlugin as a site wide setting)
3. Using an template mechanism that is called from topic object notation (similar to DBCachePlugin)

I think that templating is the most versatile mechanism in Foswiki. It allows for a default definition and skin overrides. It also gets the work done automatically and can work with existing topics and forms without having to change them. But it also occludes how things are displayed - how to improve user feedback should be a different proposal.

The default templates for the list items should be defined in the search templates.

-- ArthurClemens - 04 Jan 2011

Sven, are you willing to drive this further? Otherwise I would create a spec based on the discussion so far.

-- ArthurClemens - 21 Jan 2011

Michael pointed me to a feature of Alfresco called Aspects, which looks very powerful. If we don't want to get stuck in the "1 topic - 1 data form" model, we need to get ready for multiple topic types per topic.

What does that mean for the list views?

Let's say (in the near future) we have a list of topics changed today. There is an ExecutiveMeeting topic (containing date, place, invites), a PrizeWinningProjectTopic (containing a movie) and an XMaxPartyTopic (containing a date, place and invitation picture).

• ExecutiveMeeting has topic types: DateType, PlaceType and InviteesType
• PrizeWinningProjectTopic has topic types: MovieType
• XMaxPartyTopic has topic types: DateType, PlaceType and PictureType

Each topic type has:

• specific data fields
• several rendering methods: view, listview, listeditview

Each topic probably has a default topic type that has default rendering methods (see for a proposal TemplatingSearchResultSummaries, except that it doesn't have to be search result).

The rendering method defines how the parts of the rendering methods are drawn. This can be done using templates (easy since we have %TMPL:P{"xxx:_PREV"}%).

Topic types (and thus rendering methods) are hierarchical (probably similar to the skin template path).

So when ExecutiveMeeting is listed, it will use (from generic to specific):

• default rendering: link title, summary, image
• place rendering: adds data from place field in the metadata section (or a different section if specified)
• date rendering: adds data from date fields in the metadata section (or a different section if specified)
• invitees rendering: you get the point

-- ArthurClemens - 27 Jan 2011

Implementation proposal

The difficulty is in how to get topic data (potentially different for each topic) into the rendering process. Using a view template approach does not work because the template is read only once for each list. And using skins (for example listitem.place.tmpl) is not flexible enough when using it together with 'real' skins.

I have worked on (and prototyped) a possible implementation that now works and allows sufficient flexibility for skin authors.

In short:

1. List items are defined in search.tmpl (which is read in Search.pm).
2. Each list item has a format SEARCH:listitem defined in search.tmpl and can be overridden in search.myskin.tmpl.
3. Dynamic display variations, to render each listed item differently, are set with a new SEARCH parameter: context. This needs a small core code change in Search.pm.

See below for details.

Context parameter

In the simplest case, each list item is displayed the same, using a standard SEARCH without context parameter, or using a fixed parameter:

%SEARCH{
   "Section='Organisation'"
   type="query"
   context="place"
}%

To display items differently according to topic data, we can use tokens like $formfield to create a comma-separated list of view contexts:

%SEARCH{
   "Section='Organisation'"
   type="query"
   context="$formfield(Attributes)"
}%

Possible other tokens are $topic, $web, etc. For example:

   context="$web,$formfield(Attributes)"

Context parameter handling in Search.pm

SEARCH parameter context is a comma-separated list. In Search.pm this is stored in a variable and passed (as custom key) to sub formatResult. Each item in this list is set as context, and removed after rendering the template:

    if ( $viewContext ne '' ) {
       
        # render tokens
        $viewContext =~
s/\$formfield\(\s*([^\)]*)\s*\)/displayFormField( $topicObject, $1 )/ges;
        if ( defined $topic ) {

            $viewContext =~ s/\$web/$web/gs;
            $viewContext =~ s/\$topic\(([^\)]*)\)/Foswiki::Render::breakName( 
                                       $topic, $1 )/ges;
            $viewContext =~ s/\$topic/$topic/gs;
            $viewContext =~
s{(\$rev|\$wikiusername|\$wikiname|\$username|\$createlongdate|\$iso|\$longdate|\$date)}
                {$session->renderer->renderRevisionInfo($topicObject, $revNum, $1 )}ges;
        }

        my @contexts = split( /\s*,\s*/, $viewContext );
        foreach my $context (@contexts) {
            $session->enterContext($context);
        }

        my $text = $session->templates->expandTemplate('SEARCH:listitem');
        foreach my $context (@contexts) {
            $session->leaveContext($context);
        }

        $out = $text if $text;
    }

Template structure for list items

The default template defines 4 parts in each list item (as described earlier): image, title, summary, meta data. These can be redefined by skins (image is empty in the default definition).

These changes can be done today without changing the layout and CSS of current search results. But with minor changes, search.tmpl will have:

%TMPL:DEF{"SEARCH:format"}%%TMPL:P{"SEARCH:listitem"}%%TMPL:END%

%TMPL:DEF{"SEARCH:listitem"}%<div class='foswikiResult'>
%TMPL:P{"SEARCH:listitem:image"}%<div class='foswikiResultContents'>%TMPL:P{"SEARCH:listitem:title"}%%TMPL:P{"SEARCH:listitem:summary"}%%TMPL:P{"SEARCH:listitem:metainfo"}%</div>
</div>%TMPL:END%

%TMPL:DEF{"SEARCH:listitem:title"}%<strong>[[$web.$topic][%TOPICNAME%]]</strong>%TMPL:END%

%TMPL:DEF{"SEARCH:listitem:image"}%%TMPL:END%

%TMPL:DEF{"SEARCH:listitem:summary"}%<div class="foswikiSummary">%TEXTHEAD%</div>%TMPL:END%

%TMPL:DEF{"SEARCH:listitem:metainfo"}%<div><span class="foswikiSRRev">%REVISION% - <a href="%SCRIPTURLPATH{"rdiff"}%/%WEB%/%TOPICNAME%" rel='nofollow'>%TIME%</a></span> <span class="foswikiSRAuthor">%MAKETEXT{"by [_1] " args="%IF{"istopic '%AUTHOR%'" then="[[%AUTHOR%][$wikiname]]" else="$wikiname"}%"}%</span></div>%TMPL:END%

In a custom skin, we use the (existing) context parameter to change rendering:

%TMPL:DEF{"SEARCH:listitem:summary"}%<div class="foswikiSummary">
%TEXTHEAD% %TMPL:P{context="place" then="SEARCH:listitem:summary:place"}% %TMPL:P{context="date" then="SEARCH:listitem:summary:date"}%
</div>%TMPL:END%

-- ArthurClemens - 25 Jun 2011

i can't really work out what this new approach will do differently, and what its limitations are. It bothers me that its adding another 'view' selection mechanism - we already have a weird and hard to trace down skin path, autoview, hidden dragon thing.

re-using the application state context for a local rendering context is quite frightening - I don't think its a good idea (yet).

and lastly - this feels like a much lower level of view selection - in that place and date are data-type level, rather than DataForm level selectors - but might be misunderstanding.

i don't actually agree that the skin path can't be used for this - in exactly the same way as the VIEW_TEMPLATE is done, but I guess I should write some code to see (or did i write some and forget)

-- SvenDowideit - 26 Jun 2011

The skin path approach: let's say I have a template listitem.tmpl that defines how listed items are displayed (it may also be part of search.tmpl as illustrated above). Now I want to have different views for date items and place items. I could create listitem.place.tmpl, but that is not skinnable. For instance, listitem.place.myskin.tmpl does not work.

VIEW_TEMPLATE works on the level of the entire result set, how the list is displayed, not how list items are displayed. Moreover, you cannot have overlapping view templates, only overlapping skins.

The context is not necessarily an application state, because plugins are allowed to set the context as they wish. Of course we need to have a check mechanism that the context names do not suddenly conflict with defined names.

-- ArthurClemens - 26 Jun 2011

given this is a feature proposal - listitem.place.myskin.tmpl not currently working isn't necessarily a reason not to consider making it work (personally, i'm still confused what the problem space is - its too mixed with the solution (in both your and my proposals))

=VIEW_TEMPLATE= works on the level of the entire result set - correct, thats why i want to add a VIEWLISTITEM_TEMPLATE and EDITLISTITEM_TEMPLATE that would be defined foreach item 'type'.

yes, context is not well defined, and can be messed around in any way a perl coder wants - imo that makes it very poorly suited for end users to use to switch rendering instruction (but imo thats beside the real issue, as we can create a similar hash that isn't polluted).

I might code something up next month to show my idea

OOOOOOOOOOOhhhhh. wait a moment. your proposal embeds the to be rendered type into the SEARCH - I want to to the exact opposite. I want the rendering template foreach item to be linked to the item, so that when a SEARCH renders a list of items, it will use the right template for each one.

essentially, an AutoViewTemplatePlugin type thing for result sets that would default to something akin to

%SEARCH{
   "Section='Organisation'"
   type="query"
   context="form.name"
}%

And so, we have a similar idea - I really do need to code something up to compare.

-- SvenDowideit - 27 Jun 2011

If Foswiki::Form was more ambitious, we could move a lot of this template madness into the formfield's datatype.

I also need to get some code working to show what I mean...

-- PaulHarvey - 27 Jun 2011

In which case is a list item display not connected to search?

To connect the item display to form, simply use context="$formname". Or combine web display with form display: context="$web,$formname".

-- ArthurClemens - 27 Jun 2011

I'm getting there slowly - thanks for being patient I can now see your context param is the same as my skin setting, plus more - now I need to work out why i don't like using context, and if skin path is better or worse :/

wrt list items not connected to a search - the formating code n Search.pm is slowly becoming generalised - I want to make it trivially useful from Func so that plugin / macro authors never have to re-re-re-write code like that - but yes, I'm in future space.

-- SvenDowideit - 28 Jun 2011

Arthur actually started work on this at the CERN FoswikiCamp in 2011 - and we did find that there was only one way it can work - hopefully Arthur can remind us what that was

-- SvenDowideit - 18 Apr 2012

The prototype I had used a context param to SEARCH, which may contain a lookup value, for instance: context=$formfield(Subject).

In the render loop in Search.pm each item's context is used to call a template module for rendering.

And that is described above.

-- ArthurClemens - 18 Apr 2012

ok, so it was context that we worked out would not work - I hoped it was something else nm

-- SvenDowideit - 18 Apr 2012

It would work, but you needed to think about it. Of course there are performance concerns - caching templates to not read the template files all the time would help here.

-- ArthurClemens - 18 Apr 2012

There was some combination of SKIN path and allowing users to over-ride them that made context not work - damn, we should have written up what we discussed just before we left.

nm, I juat had an idea - If I presume DeprecateTheTmplLanguage, and AddDefaultTopicParameterToINCLUDE, we could get a listview section (or TMPL:DEF now..) from TheForm {SKIN}ViewTemplate ..... er - and like other things, maybe I can implement most of this with no code changes already?

I'll make a demo and see if it works.

-- SvenDowideit - 22 Apr 2012

Developers are no longer contributing. Changing to a rejected proposal. Objections remain after 3 years.

-- GeorgeClark