Item13820: The putBackBlocks and _putBackProtected routines are extremely inefficient.

The routines that restore removed blocks of text during the rendering process are extremely inefficient.

• It repeatedly re-scans the entire topic text for markers using a regex. 1000's of markers in some cases.
• For each hit, it expands the string in-place requiring a string copy operation.
• It processes the markers in completely random order due to perl hash reordering.

Tables, especially with EditRowPlugin requires 1000's of markers. The time in Render::_putBackProtected() was approx. 4.5 seconds. Changing the routine to re-build the page using concatenation brought the time down to 63ms. The following changes were made:

• Drive the replacement process by a scan of the topic text for markers, instead of a search of the hash keys.
• Scan using index() to locate each marker sequentially in a single pass of the data.
• Build a new page by concatenating the original text segments with the marker replacements.

-- GeorgeClark - 15 Oct 2015

Results rendering page with 400 table rows, and EditRowPlugin active:

Before: # spent 4.41s (10.1ms+4.40) within Foswiki::Render::_putBackProtected which was called 45 times, avg 97.9ms/call: 

After: # spent 62.9ms within Foswiki::Render::_putBackProtected which was called 45 times, avg 1.40ms/call:

-- GeorgeClark - 15 Oct 2015

Excellent optimization achieved.

Regexes used to be fast, alas they aren't anymore on unicode text.

I recently investigated a considerable slow-down upgrading from Foswiki 1.1.9 to 2.0.3. It turned out that the system's default perl 5.18.2 was the problem. Regexing unicode strings was back to normal speed on a more recent perl, in this case installing a local 5.22.1 did the trick.

But looking at the patch I get the impression that the new code is more efficient not only because of avoiding regexes.

Avoiding regexes is not something we should encourage as that's not the mindset when programming perl. However we have to face the situation of linux distributions that currently almost all ship a perl (mostly 5.18.2) that is not able to regex unicode text flawlessly. Foswiki will inevitably be slower by a factor of 5-20 (see also perlunicode). This also affects a lot simpler code than putBackBlocks.

-- MichaelDaum - 21 Jan 2016

Commenting is disabled while not logged in