ForEach and Search plugins combined

Hello,

I created some DataForm and attached it to each topic in my web. One of the field in DataForm is of 'select' type. Name of the field is 'QuestionCategory'.

I would like to display list of the topics in the main web with separate heading for each category.

This (category name provided as string) works for me:

%SEARCH{
      "QuestionCategory='general'"
      type="query"
      nonoise="on"
      order="$formfield(Question)"
      reverse="off"
      web="myWeb"
      format="   * [[$web.$topic][$formfield(Question)]]"
}%

but for some reason, this not:

%FOREACH{"category" in="general, other"}%

---+++++ $category

%SEARCH{
      "QuestionCategory='$category'"
      type="query"
      nonoise="on"
      order="$formfield(Question)"
      reverse="off"
      web="myWeb"
      format="   * [[$web.$topic][$formfield(Question)]]"
}%

%NEXT{"category"}%

I've tried to modify my query many times as well as use Encode macro (to provide category with ' characters), but without any results. Only heading is properly displayed, but no search results are produced.

-- MateuszKDzior - 07 Jan 2015

The problem is that the SEARCH macro is being rendered at same time as the FOREACH macro. As noted in ForEachPlugin docs: "The body of the loop may need to delay expansion: use $percnt to replace % if necessary. "

Therefore, what you need is this (untested):

%FOREACH{"category" in="general, other"}%

---+++++ $category

$percntSEARCH{
      \"QuestionCategory='$category'\"
      type=\"query\"
      nonoise=\"on\"
      order=\"$formfield(Question)\"
      reverse=\"off\"
      web=\"myWeb\"
      format=\"   * [[$web.$topic][$formfield(Question)]]\"
}$percnt

%NEXT{"category"}%

For your particular use case, you might also consider using the VarFORMAT macro that is part of Foswiki core. Here's your example using that macro:

%FORMAT{
   "general,other" 
   type="string" 
  format="---+++++ $item

     $percntSEARCH{
         \"QuestionCategory='$item'\"
         type=\"query\"
         nonoise=\"on\"
         order=\"$formfield(Question)\"
         reverse=\"off\"
         web=\"myWeb\"
         format=\"   * [[$web.$topic][$formfield(Question)]]\"
   }$percnt"
}% 

For more complex use cases along these lines, I'd recommend considering FilterPlugin. It may appear a bit intimidating initially but it does a lot and I find I use it all the time.

Both VarFORMAT and FilterPlugin follow normal macro expansion rules which this one doesn't (which is the real source of your problem).

-- LynnwoodBrown - 09 Jan 2015

Thank you for detailed answers.

However - suppose that I wish to combine this with Twisty plugin (Support.Question1550)

Why my example doesn't work?

%FOREACH{"category" in="general, other"}%

---+++++ !$category

$percntSEARCH{
        "QuestionCategory='$category'"
        topic="FAQ*"
        type="query"
        nonoise="on"
        order="$formfield(Question)"
        reverse="off"
        web="FAQ"
        format="$percntTWISTY{ showlink=\"Show $topic\" hidelink=\"Hide\"}$percnt $percntINCLUDE{$web.$topic}$percnt $percntENDTWISTY$percnt"
}$percnt

</br><span>--------------------------------------</span>
   
%NEXT{"category"}%

-- MateuszKDzior - 12 Jan 2015

Moreover, I would like to read categories from my form (which I have defined in FAQ/QuestionForm).

Unfortunately, something is wrong - even if I place following lines outside any macro, no one produce any results:

%QUERY{"FAQ.QuestionForm/form.QuestionCategory"}% 
%QUERY{"FAQ/QuestionForm/form.QuestionCategory"}% 
%QUERY{"FAQ/QuestionForm/QuestionCategory"}%

Could you tell me why?

My QuestionForm is defined as follows:

<noautolink>
| *Name*  | *Type* | *Size* | *Values* | *Tooltip message* | *Attributes* | 
| Question | text | 100 | | Provide question here. It will be used during topics searching. | M | 
| !QuestionCategory | select | 1 | ,general, specific, other | | | 
| Related Topics | textboxlist | | | | 
| Comments | text | 500 | | | | 
</noautolink>

-- MateuszKDzior - 12 Jan 2015

One thing... I don't think you should use the exclamation in the QuestionCategory name in the table. Better to create a topic called QuestionCategory, or just let it link to a missing page.

-- GeorgeClark - 12 Jan 2015

Ok, let me address your several comments/questions separately. First of all, the reason why your example that includes the twisty doesn't work is because the twisty is inside the search format so it's rendering must be delayed (escaped) again. Same with the INCLUDE. You can see what this would look like in my final solution below.

Next, regarding your desire to pull the category values from the data form, VarQUERY is not designed to pull values from data form definition topics. It's designed to query values from the form as used by a topic. There is a way to compose a SEARCH which can return the values from the form definition topic. Before I describe the SEARCH, I'd like to agree with GeorgeClark: don't use '!QuestionCategory' as it confuses several things. You can simple use "Question Category" as the field name. It stills get searched as 'QuestionCategory'. In any case, the SEARCH for getting that fields values would look something like this:

%SEARCH{"QuestionForm" 
  scope="topic" 
  format="$pattern(.*?\| Question Category \|.*?\|.*?\|(.*?)\|.*)"
  nonoise="on"
}%

Using the $pattern option for FormattedSearch is tricky because it requires some understanding of RegularExpression. An easier approach for this kind of thing is using FlexFormPlugin.

One other little detail: to reference a field for the "order" parameter in the SEARCH, don't put "$" in front of "formfield(Question)".

OK, let's put the whole thing together:

%FOREACH{"category" 
       in="%SEARCH{"QuestionForm" 
                    scope="topic" 
                    format="$pattern(.*?\| Question Category \|.*?\|.*?\|(.*?)\|.*)"
                    nonoise="on"
              }%"
}%

---+++++ !$category

$percntSEARCH{
        "QuestionCategory='$category'"
        topic="FAQ*"
        type="query"
        nonoise="on"
        order="formfield(Question)"
        reverse="off"
        web="FAQ"
        format="$dollarpercntTWISTY{ showlink=\\"Show $topic\\" hidelink=\\"Hide\\"}$dollarpercnt $dollarpercntINCLUDE{$web.$topic}$dollarpercnt $dollarpercntENDTWISTY$dollarpercnt"
}$percnt

</br><span>--------------------------------------</span>
   
%NEXT{"category"}%

You're probably wondering why the SEARCH to get the form values isn't escaped. The reason for that is it does not depend on the output of the FOREACH macro to work so it can be rendered before the FOREACH macro.

Hope this helps. BTW, your additional questions on this subject are wandering pretty far from your original question. If you have further questions about other aspects of what you're trying to accomplish, I'd recommend posting separate questions.

-- LynnwoodBrown - 12 Jan 2015

Thank you but something is still wrong - even without Twisty part. Maybe it will be better/easier to avoid ForEach macro, but since you told me no to ask about other aspects of my question, I'm not asking about that

---

First of all, I changed '!QuestionCategory' to Question Category. As I can see your search, produce correct output. Separate:

%SEARCH{"QuestionForm" 
                    scope="topic" 
                    format="$pattern(.*?\| Question Category \|.*?\|.*?\|(.*?)\|.*)"
                    nonoise="on"
              }%

produce:

category 1,general other, category, 2, category 3, old category (before category 1)

---

Now, when I simply replace list of categories with your search part:

%FOREACH{"category" 
       in="%SEARCH{"QuestionForm" 
                    scope="topic" 
                    format="$pattern(.*?\| Question Category \|.*?\|.*?\|(.*?)\|.*)"
                    nonoise="on"
              }%"
}%

---+++++ !$category

$percntSEARCH{
        "QuestionCategory='$category'"
        topic="FAQ*"
        type="query"
        nonoise="on"
        order="formfield(Question)"
        reverse="off"
        web="FAQ"
        format="   * [[$web.$topic][$formfield(Question)]]"
}$percnt

</br><span>--------------------------------------</span>
   
%NEXT{"category"}%

there's no topics displayed for my last category. It looks like only this part:

---+++++ !$category

of code has been executed for the last category. So maybe it will be better to assign search result to separate variable and then execute ForEach?

-- MateuszKDzior - 14 Jan 2015

At least we're making progress! Just to be clear, is "old category (before category 1)" the complete string of your last category? It could be that somehow the parentheses are playing mischief but I can't see why that would be. You might try adding zeroresults="No records found" or something like that to your search just to get confirmation that the search is happening. Other than that, I don't know what to suggest. If the FOREACH is working otherwise with the search, I can't see any benefit from replacing with macro of the same output.

-- LynnwoodBrown - 14 Jan 2015

Okay, now I get it...I have an additional space at the end of last category and it seems that it cause this issue.

I changed my QuestionForm - from:

| Question Category | select | 1 | category 1,general other, category, 2, category 3, old category (before category 1) | | | 

to (remove spaces)

| Question Category | select | 1 |category 1,general other,category,2,category 3,old category (before category 1)| | | 

and now ForEach produce results based on Search.

-- MateuszKDzior - 16 Jan 2015

...but there's still an issue with ForEach version which embed Twisty

I thought, that I should simply replace $dollarpercnt with $dollar$percnt, but it doesn't help.

Will it be possible/easier to generate input for Twisty in different topic (since we already now how to do that) and then (but I guess I need to create another topic for that?).

-- MateuszKDzior - 16 Jan 2015

The expansion order of Foswiki macros and the appropriate way to escape/delay them is tricky. Going back up to the example where you had a twisty inside the search inside the FOREACH, $dollarpercnt, was the correct format. $percnt delays expansion of the macro one level (as in the case of the SEARCH inside FOREACH. For the next level, we use $dollar to escape the "$" in $percnt. So $dollarpercnt would be expanded as part of the the SEARCH results. I try not to go more than two levels of escaping in this way as is just gets uuggly (e.g. $dollardollarpernt). Did that example not work? I'm pretty sure it should. The syntax you suggested of $dollar$percnt would merely expand to "$%" at the first level expansion (i.e. under the FOREACH).

One way to get around multiple levels of nesting is to use includes, as you suggested, either to another topic or to another section within the same topic. The nice thing about VarINCLUDE is that macros in the included content gets completely expanded before being inserted, therefore you don't have to worry about escaping them. It just takes practice (and trial and error) to get an understanding about the best strategies to use macro escaping and INCLUDES to achieve your purpose.

-- LynnwoodBrown - 16 Jan 2015

Commenting is disabled while not logged in