Item2276: Searching recursively does not sort correctly

When I search recursively, I expect the ordering defined by the order parameter to apply across all the topics found by the search. instead, it only applies within the scope of each individual web. For example, I create three subwebs, A, B and C, and add a topic, 'TheTopic' to each. I then add a form with a field called "Order" to each TheTopic, and set the value to 2, 3, and 1 for subwebs A, B, and C respectively. I then invoke:

%SEARCH{"Order!=''"
 type="query"
 topic="TheTopic"
 recurse="on"
 nonoise="on"
 order="formfield(Order)"
 format="$web"
 separator=","}%

I expect the order of the found topics to be C, A, B, but instead see A, B, C. This is because the code is sorting the results within each subweb, and then applying the alphanumeric natural ordering to the web names to create the final result.

I added a unit test to trunk, to illustrate the behaviour.

We have gone a long time without this being noticed, so I guess it isn't Urgent, though it is for me

This bug exists all the way back to (tm)wiki. I believe it should be fixed for 1.1.

-- CrawfordCurrie - 21 Oct 2009

One of the oldest known bugs of SEARCH. Fixing rejected on the old project saying this was the way sort+limit across webs is to be "optimized".

-- MichaelDaum - 21 Oct 2009

OK, that's useful to know. The rejection is obviously bollocks, I move that this be fixed.

-- CrawfordCurrie - 21 Oct 2009

I do not remember anyone rejecting the desired spec in the old or new project.

The argument was that current behaviour was implemented based on performance. Ie the one giving this argument gad bad experience with another solution.

So whenever anyone tries to fix this, please pay attention to performance.

But don't worry about the spec. Sorting all the results as one result set is what we want.

-- Kenneth Lavrsen - 24 Oct 2009

this is not a side effect of the performance pre-optimisation, this is (sadly) due to the 10 year history that all SEARCH results are to be rendered one web at a time. the SEARCH loop is (though I'm slowly working through removing this) implemented as

foreach web {
   print web header
   sort the topics
   foreach topic {
      foreach multi-hit {
         render topic info
      }
   }
   print web footer
}

I am going to continue to work towards removing this, but we will experience some incompatibilities somewhere - ie, we need many more unit tests

-- SvenDowideit - 29 Oct 2009

Thanks Sven. My solution would have been major surgery on Search.pm, which would likely have conflicted with your work, so I'm staying paws-off on this one. Will continue to add tests as and when I see the need for them, tho.

-- CrawfordCurrie - 29 Oct 2009

Moving this up from Normal to Urgent so it stays on Sven's radar for his search work.

-- CrawfordCurrie - 16 Mar 2010

its not likely to leave my radar - as its what i started last week, and will be working on next week

-- SvenDowideit - 16 Mar 2010

this is now possible using the groupby=none secret sauce.

-- SvenDowideit - 01 Apr 2010