Feature Proposal: Refactor format rendering to use ResultSets (adds paging, consistency and speed)

Motivation

We've known for a long time that we need to add scalable paging to our Searches, GROUPs and such. I propse we do this, by extracting the format= code from all the TML, and move that code to take a Foswiki Iterator (the basis of what will become ResultSets). (and we expose that via Func). That way, we create one consistent formatting system, and will be able to add paging in a generic way to all tags.

AddUSERSLISTandGROUPSLIST is another example where we need to implement Paging - the Joomla and LDAP use cases are known to have way more users than should be displayed in one hit.

Description and Documentation

Extract the formatting stage from SEARCH and other tags, re-write it to use FoswikiIterators, with filtering, and late evaluation of Iterators where possible.

Crawford and I have been working towards this over the last years, but it needs a final push (ok, quite a lot of work) to get it completed.

Include adding and merging the TOM specifiers so that we have the same syntax in If, query SEARCH and format strings. see AddAttachmentsVarToFormattedSearch

probable outcomes

• speedup oportunities - via late evaluation of iterators
• consistent paging and formating parameters and code that will be reuseable by plugins
• Would enable ResultSets that could then be used for nested operations
• %FORMAT{}% TML func that can be used to apply a format to any list of results - whether that is a SEARCH result, GROUP list, Saved Query Result - basically informs Foswiki howto render ResultSets.

Examples

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

-- Contributors: SvenDowideit - 15 Dec 2007

Discussion

Don't forget to add the DateOfCommitment I added 2007-12-15.

I support this proposal. It is a very much needed feature. Both for the core and for plugins.

-- KennethLavrsen - 17 Dec 2007

What does "using FoswikiIterators" mean for the end user who's used to type format="..." ? I don't know much about the concept of iterators so any clarification would be appreciated

-- CarloSchulz - 17 Dec 2007

for the user, the only thing they might notice, is that all the format= statements use a common syntax. Right now, some Macros don't support $n some use $date - that sort of thing.

mostly, no change except for consistency, and a common paging scheme for all mulitple result tags.

-- SvenDowideit - 17 Dec 2007

An iterator would also be useful in templates. We could do away with SPLIT, and simplify code for lists of buttons.

For iterating over search results, see the proposal in SearchResultsPagination. Note the sort step and the need to have iterating controls.

-- ArthurClemens - 17 Dec 2007

Aha, what Arthur mentions brings up yet another proposal that will be more noticable... ResultSets - which will thus give us the need to add a %FORMAT{}% function, that will use a specified (or named) result (aka, iterator, setting, search, GROUP)

-- SvenDowideit - 21 Dec 2007

Seems all are happy. Accepted by 14 day rule.

-- KennethLavrsen - 02 Jan 2008

another point - could do with the raw='on' working everywhere - essentially it would retreive and render the format, and then would prevent foswiki from rendering it further.

-- SvenDowideit - 01 Jun 2008

I don't understand that. raw=on means "use a specific template for this page" to the view script. I had been thinking that TinyMCEPlugin should respect "raw=on" as synonymous with nowysiwyg=1. But you seem to be suggesting something else...

-- CrawfordCurrie - 01 Jun 2008

no, i think most users see raw=on means show me the unrendered version. so TML{format="$web.$topic" raw=on} would render something like Web.TopicName

-- SvenDowideit - 01 Jun 2008

This is essential to ideas like RelaxRegisterTagHandlerCallingContext

-- SvenDowideit - 05 Sep 2008

after discussion, we're going to rename %FORMAT{}% to %FOREACH{}% , and AddAngleBracketsToFormatTokens ad other older discussions remind me that I wanted to add a $tom(TOM) (or call it eval() or something) and $include(topic, section) which can be used to expand text after SEARCH / IF / to give us a way to evaluate outside->in.

-- SvenDowideit - 05 Sep 2008

I note that there is a ForEachPlugin that has a tag name clash with the proposed %FOREACH{}% given here

-- JulianLevens - 08 Oct 2009

The ForEachPlugin was part of the discussion that led to FOREACH as the macro as the feature very much overlap.

At least this is how I recall it. The discussion was on IRC and in some other topics here in Development web.

-- KennethLavrsen - 23 Mar 2010

See also Tasks.Item8614 ForEachPlugin fails when used with SpreadSheetPlugin

-- MartinCleaver - 23 Mar 2010

I guess what Martin means is that when Sven implements this he should check the bug reported in the 8614 and the support question that the bug item refers to.

-- KennethLavrsen - 23 Mar 2010

The task related to this is closed. Developer has left, so marking this merged to core.

-- GeorgeClark - 19 Nov 2015

Commenting is disabled while not logged in