Item13410: Sorting of Search results is incorrect when paging is applied.

Hi,

I build a table based on SEARCH. As many lines are expected, the Table is multipage, which suppose to be sorted, e.g. by the year (or date). If all outputs are on the same one page - sorting is fine, however, if they are split over several pages, sorting is not what one would expect, i.e. I have years 2009, 2010, 2011, 2012, 2013, 2014. If it is said pagesize=10, i.e. all entries are on one page: sorting is fine. If I say pagesize=2, then sorting is as following: page 1: (2011, 2012) page 2: (2009, 2010) page 3: (2013, 2014) While of course, I would expect (2009,2010), (2011,2012), (2013,2014) example on the Foswiki.org: http://foswiki.org/Sandbox/MySearchTable

-- ValentinKozlov - 19 May 2015

There are two sorts going on here, and I don't believe that this is fixable, or even a bug. You've asked SEARCH to sort on a formfield and return a subset of the results, So it has done it's job and returned 2 items in one order. They are then presented in table format, which you request in a different order from the SEARCH.

The SEARCH has no idea that it's embedded in a table. It expands and returns a subset of the total results, Only then during rendering does the table sorting happen. (In the latest version of foswiki, table sorting is often done client side in javascript). The only way to have consistent table sorting is to sort only the rows presented to the table.

Setting to No Action. There's no way to fix this.

-- GeorgeClark - 26 Sep 2016

I think you right, this is actually not a TablePlugin problem though, it worked perfectly fine in TablePlugin 1.132 and FW1.1.3.

Now, I removed the Table and left only SEARCH (see bottom of the http://foswiki.org/Sandbox/MySearchTable page), $pagesize is a parameter of SEARCH, I do sort by $formfield, the result is as before disappointing: page 1: (2011, 2012) page 2: (2009, 2010) page 3: (2013, 2014)

Could you tell me then, what is wrong with SEARCH? Hopefully not in another 1,5 year ....

-- ValentinKozlov - 27 Sep 2016

Unfortunately this seems to be an "optimization" of search that is not very helpful. The SEARCH processing can be extremely slow, and one way to speed things up in a paged environment is to only process enough records to satisfy the page size in each response. While that speeds up the search, it means that the order is not applied to the entire set of results. It really requires a design change to delegate filtering and sorting to a deeper layer of code.

I'm re-categorizing this as a bug against SEARCH, however as it's a major modification to the internals, it's rather unlikely to be fixed short term.

-- GeorgeClark - 28 Sep 2016

Downgrading this to Normal. It's been around a long time, and it is not something that will be fixed in a simple patch. Its too complex to block a release.

-- GeorgeClark - 13 May 2017

Commenting is disabled while not logged in