You are here: Foswiki>Development Web>ReleaseProcess>FeatureRequest>EasierNestedSearch (08 Jul 2010, CrawfordCurrie)Edit Attach

Feature Proposal: Reduce $dollardollar

Motivation

Very hard to write nested searches, especially as they become deep

Description and Documentation

Allow
  • $(1) for $dollar
  • $(2) for $dollardollar
  • $(n) for $dollardollar...dollar (i.e dollar n times)
  • $(16) is the maximum

Examples

From FormattedSearch 
%SEARCH{ "culture" web="System" format="   * System.$topic is referenced by:$n      * $percntSEARCH{ \"$topic\" web=\"System\" format=\"System.$dollartopic\" nosearch=\"on\" nototal=\"on\" separator=\", \" }$nop%" nosearch="on" nototal="on" }%
Result:

Could become: 
%SEARCH{ "culture" web="System"
 nosearch="on" nototal="on"
 format="   * System.$topic is referenced by:$n      * \
$(1)percntSEARCH{ $(1)quot$topic$(1)quot web=$(1)quotSystem$(1)quot
 nosearch=$(1)quoton$(1)quot nototal=$(1)quoton$(1)quot separator=$(1)quot, $(1)quot
 format=$(1)quotSystem.$(1)topic\
)$(1)quot }$(1)percnt\
"}%
Not a great deal easier, but adding the next level, you get: 
%SEARCH{ "culture" web="System"
 nosearch="on" nototal="on"
 format="   * System.$topic is referenced by:$n      * \
$(1)percntSEARCH{ $(1)quot$topic$(1)quot web=$(1)quotSystem$(1)quot
 nosearch=$(1)quoton$(1)quot nototal=$(1)quoton$(1)quot separator=$(1)quot, $(1)quot
 format=$(1)quotSystem.$(1)topic\
in turn by (\
$(2)percntSEARCH{ $(2)quot$(1)topic$(2)quot web=$(2)quotSystem$(2)quot
  nosearch=$(2)quoton$(2)quot nototal=$(2)quoton$(2)quot separator=$(2)quot, $(2)quot
  format=$(2)quotSystem.$(2)topic\
$(2)quot }$(2)percnt\
)$(1)quot }$(1)percnt\
"}%
Notice that the next level is almost a repeat of the prior level and just increment the number.

A more extensive example comes from existing code on our TWiki site 
   * Local TERMS = type="regex" scope="text" nonoise="on" separator="$n"
   * Local DO = %TERMS% search="META:FORM.*?name=\"TW4TeamForm\";META:TOPICPARENT.*?name=\"
   * Local END = \""
   * Local FORMAT = $topic $percntSEARCH{$quotMETA:FORM.*?name=.*?UserForm.;name=.TeamName.*?value=.$topic\"$quot $percntTERMS$percnt header=\"$n\" format=\"                        * icon:person $dollartopic $dollarformfield(Profession)\"}$percnt

%RENDERLIST{"TW4"}%
%~~ SEARCH{%DO%%URLPARAM{"start" default="WebHome"}%%END%
~~~     format="   * %FORMAT%
*~~     $n$percntSEARCH{$percntDO$percnt$topic$percntEND$percnt
~~~        format=$quot      * $percntFORMAT$percnt
*~~        $n$dollarpercntSEARCH{$dollarpercntDO$dollarpercnt$dollartopic$dollarpercntEND$dollarpercnt
~~~           format=$dollarquot         * $dollarpercntFORMAT$dollarpercnt
*~~           $n$dollardollarpercntSEARCH{$dollardollarpercntDO$dollardollarpercnt$dollardollartopic$dollardollarpercntEND$dollardollarpercnt
~~~              format=$dollardollarquot            * $dollardollarpercntFORMAT$dollardollarpercnt
*~~              $n$dollardollardollarpercntSEARCH{
*~~                      $dollardollardollarpercntDO$dollardollardollarpercnt
*~~                      $dollardollardollartopic$dollardollardollarpercntEND$dollardollardollarpercnt
~~~                 format=$dollardollardollarquot               * $dollardollardollarpercntFORMAT$dollardollardollarpercnt
*~~                 $n$dollardollardollardollarpercntSEARCH{
*~~                         $dollardollardollardollarpercntDO$dollardollardollardollarpercnt
*~~                         $dollardollardollardollartopic$dollardollardollardollarpercntEND$dollardollardollardollarpercnt
~~~                    format=$dollardollardollardollarquot                  * $dollardollardollardollarpercntFORMAT$dollardollardollardollarpercnt
*~~                    $dollardollardollardollarquot
*~~                 }$dollardollardollardollarpercnt
*~~                 $dollardollardollarquot
*~~              }$dollardollardollarpercnt
*~~              $dollardollarquot
*~~           }$dollardollarpercnt
*~~           $dollarquot
*~~        }$dollarpercnt
*~~        $quot
*~~     }$percnt
*~~     "
~~~  }%
Becomes the much simpler (not tested): 
   * Local TERMS = type="regex" scope="text" nonoise="on" separator="$n"
   * Local DO = %TERMS% search="META:FORM.*?name=\"TW4TeamForm\";META:TOPICPARENT.*?name=\"
   * Local END = \""
   * Local FORMAT = $topic $percntSEARCH{$quotMETA:FORM.*?name=.*?UserForm.;name=.TeamName.*?value=.$topic\"$quot $percntTERMS$percnt header=\"$n\" format=\"                        * icon:person $(1)topic $(1)formfield(Profession)\"}$percnt

%RENDERLIST{"TW4"}%
%~~ SEARCH{%DO%%URLPARAM{"start" default="WebHome"}%%END%
~~~     format="   * %FORMAT%
*~~     $n$percntSEARCH{$percntDO$percnt$topic$percntEND$percnt
~~~        format=$quot      * $percntFORMAT$percnt
*~~        $n$(1)percntSEARCH{$(1)percntDO$(1)percnt$(1)topic$(1)percntEND$(1)percnt
~~~           format=$(1)quot         * $(1)percntFORMAT$(1)percnt
*~~        $n$(2)percntSEARCH{$(2)percntDO$(2)percnt$(2)topic$(2)percntEND$(2)percnt
~~~           format=$(2)quot            * $(2)percntFORMAT$(2)percnt
*~~        $n$(3)percntSEARCH{$(3)percntDO$(3)percnt$(3)topic$(3)percntEND$(3)percnt
~~~           format=$(3)quot               * $(3)percntFORMAT$(3)percnt
*~~        $n$(4)percntSEARCH{$(4)percntDO$(4)percnt$(4)topic$(4)percntEND$(4)percnt
~~~           format=$(4)quot                  * $(4)percntFORMAT$(4)percnt
*~~        $(4)quot
*~~        }$(4)percnt
*~~        $(3)quot
*~~        }$(3)percnt
*~~        $(2)quot
*~~        }$(2)percnt
*~~        $(1)quot
*~~        }$(1)percnt
*~~        $quot
*~~     }$percnt
*~~     "
~~~  }%
That's easier to follow. Note that there is no $dollar at all in this example. $(1) is the same as $dollar. Also note that $(1)dollar is equivalent to $(2) (or $dollardollar) etc

Impact

WhatDoesItAffect:

Implementation

One line added to: 
sub expandStandardEscapes {
    my $text = shift;
# New line here:
    $text =~ s/\$[(]([1-9]|[1][1-6])[)]/ '$' . ($1 == 1 ? '' : '(' . ($1 - 1) . ')' )/gose;
    
    $text =~ s/\$n\(\)/\n/gos;    # expand '$n()' to new line
    $text =~ s/\$n([^$regex{mixedAlpha}]|$)/\n$1/gos;  # expand '$n' to new line
    $text =~ s/\$nop(\(\))?//gos;      # remove filler, useful for nested search
    $text =~ s/\$quot(\(\))?/\"/gos;   # expand double quote
    $text =~ s/\$percnt(\(\))?/\%/gos; # expand percent
    $text =~ s/\$dollar(\(\))?/\$/gos; # expand dollar
    return $text;
}
-- Contributors: JulianLevens - 13 Sep 2009

Discussion

interesting idea..

now, I wonder if we can go another step further? - by adding a $nest() operator that then allows each nesting to be like a function call.

Originally I was thinking of doing this sort of thing using sectional includes, but that also goes a bit wonky

as a minor eg, I would write 
   * Local FORMAT = $topic $percntSEARCH{$quotMETA:FORM.*?name=.*?UserForm.;name=.TeamName.*?value=.$topic\"$quot $percntTERMS$percnt header=\"$n\" format=\"                        * icon:person $(1)topic $(1)formfield(Profession)\"}$percnt
as 
%STARTSECTION{"format"}%%TNAME% %SEARCH{"META:FORM.*?name=.*?UserForm.;name=.TeamName.*?value=.%TNAME%\" %TERMS% header=\"$n\" format=\"                        * icon:person $topic $formfield(Profession)\"}%%ENDSECTION{"FORMAT"}%
and then use it by 'calling' 
   format="$percntINCLUDE{"%BASEWEB%.%BASETOPIC%" TNAME="$topic"}$percnt"
and the proposal I have on the table somewhere for the Format work I'm doing in trunk would look more like: 
   format="$include("%BASEWEB%.%BASETOPIC%" TNAME="$topic")"
basically, I prefer to use sectional includes so that I can re-use parts at different nesting levels.

-- SvenDowideit - 14 Sep 2009

While I approve wholeheartedly of what you are trying to do here, I can't help feeling there has to be a simpler way of expressing it. For example, if we had parameterisable macros, you might write (similar to Sven's transclusion approach): 
  * Local SUBSEARCH(a) = %SEARCH{ "%a%" web="System" format="System.$topic" nosearch="on" nototal="on" separator=", " }%
%SEARCH{ "culture" web="System" format="   * System.$topic is referenced by:$n      * $percntSUBSEARCH{a=\"$topic\"}$percnt" nosearch="on" nototal="on" }%
Look, no $dollars. If we handled string escapes properly, this would reduce to: 
  * Local SUBSEARCH(a) = %SEARCH{ "%a%" web="System" format="System.$topic" nosearch="on" nototal="on" separator=", " }%
%SEARCH{ "culture" web="System" format="   * System.$topic is referenced by:$n      * \%SUBSEARCH{a=\"$topic\"}\%" nosearch="on" nototal="on" }%
Aside: Heaven forfend that we support both sexes of quote, which would allow us to reduce even further to: 
  * Local SUBSEARCH(a) = %SEARCH{ "%a%" web="System" format="System.$topic" nosearch="on" nototal="on" separator=", " }%
%SEARCH{ "culture" web="System" format="   * System.$topic is referenced by:$n      * \%SUBSEARCH{a='$topic'}\%" nosearch="on" nototal="on" }%
-- CrawfordCurrie - 14 Sep 2009

Whenever possible organize your wiki apps into separate TopicFunctions where you keep reusable bits on topics of their own with their implementation in a %STARTINCLUDE%... %STOPINCLUDE% section. Sub searches are then called via $percntINCLUDE{\"RenderSubResults\" param1 param2 param3}$percnt. Use a more speaking name that reflects the intend of the called function. Keep escape levels to a max of two so that you will never see more complex expressions than $dollarpercent. Or the other way around: whenever you feel the need of a format string that has got a dollardollar in it then take this as an indicator for your application not being well-structured.

A best practice prototype is: 
%SEARCH{
  format="$percntINCLUDE{\"RenderResultItem\" ... }$percnt"
}%
Move all complexity into RenderResultItem and proceed within there along the same lines. Use each TopicFunction to docu what it does.

-- MichaelDaum - 14 Sep 2009

Thanks for the feedback, I'll look into getting these alternatives working and understanding how it hangs together.

Initial thoughts are:
  1. The role-of thumb to consider $dollardollar in an application as one that needs redesign is probably valid. However, my second case scenario is a classic hierarchical search (to create an org-chart) as often required from SQL sometimes with special support in the SQL. I also know that Crystal reports has a feature just for this case. There is an alternative database design "Nested Sets", but that will not be easy to implement as DataForms and has other drawbacks.
  2. Pushing my code into a sub-topic is not necessarily a redesign. It's fine to choose that for design reasons, quite another because a limitation or your tool (more or less) forces you to.
  3. I am not desperate for this and if I can end with an easy to maintain alternative that's OK for the moment. I'm optimistic that other developments will eventually provide even better solutions - including performance.
  4. I could look at capturing the above into a NestedFormattedSearch topic if that makes sense.
  5. My code is explicitly limited to 5 levels, the alternatives look recursive but I believe end when a search returns no hits - and hence does not format another search. Just need to ensure that's right especially if I docu this.
-- JulianLevens - 14 Sep 2009

On IRC, a user pointed out http://mojomojo.org/development/discussion/mojomojo_plugin_syntax#XML

-- PaulHarvey - 08 Jan 2010

ChangeProposalForm edit

TopicSummary Reduce $dollar
CurrentState UnderInvestigation
CommittedDeveloper
ReasonForDecision None
DateOfCommitment
ConcernRaisedBy
BugTracking
RelatedTopics UseSyntaxToChangeEvaluationOrder, HereDocumentSyntaxForMacros FeaturesOfSEARCH
PlannedFor
Edit  | Attach | Print version | History: r9 < r8 < r7 < r6 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r9 - 08 Jul 2010, CrawfordCurrie
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy