Feature Proposal: Add different sorting behaviour(s) to search output

Motivation

Right now, order="..." only does text-based sort (at least when not using date-based sorting). In many cases, it would be nice to sort numerically.

Description and Documentation

MichaelTempest suggested on IRC that %SEARCH and family need a sortas="..." or "compareas="..." option - with predefined comparisons for numbers, number-like-text such as MartinKaufmann's , text, and dates - and plugins should be able to add new comparisons.

Examples

In my case, I'd like to sort using document IDs. I've got a SEARCH with order="formfield(DocumentID)" and the ID is a number. To make it more complicated (and to make it harder for a generic solution) the IDs are not plain numbers but of the form "ID 1-1", "ID 1-2" etc. Right now, the sorting leads to the following result: ID 1-1, ID 1-10, ID 1-11, ID 1-2, ID 1-3, ...

Below is MichaelTempest's patch to lib/Foswiki/Search.pm (based on Foswiki 1.0.9):

--- Search.pm.orig      2010-01-17 15:16:20.000000000 +0100
+++ Search.pm   2010-07-07 17:28:18.000000000 +0200
@@ -1122,17 +1122,28 @@

 # RE for a full-spec floating-point number
 my $number = qr/^[-+]?[0-9]+(\.[0-9]*)?([Ee][-+]?[0-9]+)?$/s;
+my $dashedNumber = qr/(\d+)((?:-\d{1,3})+)$/;

 sub _compare {
-    if ( $_[0] =~ /$number/o && $_[1] =~ /$number/o ) {
-
-        # when sorting numbers do it largest first; this is just because
-        # this is what date comparisons need.
-        return $_[1] <=> $_[0];
-    }
-    else {
-        return $_[1] cmp $_[0];
-    }
+  if ($_[0] =~ /$dashedNumber/o) {
+    my $leading_0 = $1;
+    my $trailing_0 = $2;
+    if ($_[1] =~ /$dashedNumber/o) {
+      my $leading_1 = $1;
+      my $trailing_1 = $2;
+      $trailing_0 =~ s/-(\d+)/sprintf '%03d', $1/ge;
+      $trailing_1 =~ s/-(\d+)/sprintf '%03d', $1/ge;
+      my $number_0 = "$leading_0.$trailing_0";
+      my $number_1 = "$leading_1.$trailing_1";
+      return $number_1 <=> $number_0;
+    }
+  }
+  if ($_[0] =~ /$number/o && $_[1] =~ /$number/o) {
+    return $_[1] <=> $_[0];
+  }
+  else {
+    return $_[1] cmp $_[0];
+  }
 }

I can see that it's quite hard to come up with a generic solution which doesn't require a separate if for each format.

---

Ideally, TablePlugin should be able to (re-)use the same sorting logic, so that sorting order is the same regardless of what does the sorting. Furthermore, there should be a rest handler so that JS stuff (e.g. JQTablePlugin) may sort things consistently, too.

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

-- Contributors: MartinKaufmann, MichaelTempest

Discussion

I am under the impression that we should be moving toward a generic header/format/footer solution viz. VarFOREACH where something like a SEARCH would actually only output a comma-separated list of items and the FOREACH takes care of how it should be rendered.

I would like to hear from the Foswiki::Meta and search tinkerers (CrawfordCurrie, SvenDowideit) about where they feel customised sorting belongs: in FOREACH, or the the thing feeding the list into FOREACH (I suspect the latter)? Or maybe a 3rd macro that only does sorting?...

Incidentally, FormFieldListPlugin might be extensible enough to provide this functionality as an interim solution.

-- PaulHarvey - 07 Jul 2010

Sorry, I don't have time right now to add what i want to here, other than to say, yes, a generalised pluggable sorting mechanism is in the cards, and is needed to acheive the kind of thing you mention.

It would be nice to have this added to the multi-sorting missing feature summary that Crawford started after I started a few other sorting related feature proposals :/

or better yet, to refactor them all into one proposal (deleting the others) cos its getting really hard to remember them all.

please also note that without unit tests and documentation, patches like this aren't much help, other than something to critique

-- SvenDowideit - 08 Jul 2010

SupportMultiKeySorting is the related proposal we were thinking of.

-- CrawfordCurrie - 08 Jul 2010

I wrote that patch for Martin's specific requirement. It is most certainly not generic

I definitely support Sven and Crawford's suggestion to remove this proposal and transfer the detail to SupportMultiKeySorting.

-- MichaelTempest - 08 Jul 2010