Item13820: The putBackBlocks and _putBackProtected routines are extremely inefficient.
Priority: Enhancement
Current State: Closed
Released In: 2.0.3
Target Release: patch
Applies To: Engine
Component: FoswikiRender, Performance
Branches: master
The routines that restore removed blocks of text during the rendering process are extremely inefficient.
- It repeatedly re-scans the entire topic text for markers using a regex. 1000's of markers in some cases.
- For each hit, it expands the string in-place requiring a string copy operation.
- It processes the markers in completely random order due to perl hash reordering.
Tables, especially with
EditRowPlugin requires 1000's of markers. The time in
Render::_putBackProtected()
was approx. 4.5 seconds. Changing the routine to re-build the page using concatenation brought the time down to 63ms. The following changes were made:
- Drive the replacement process by a scan of the topic text for markers, instead of a search of the hash keys.
- Scan using
index()
to locate each marker sequentially in a single pass of the data.
- Build a new page by concatenating the original text segments with the marker replacements.
--
GeorgeClark - 15 Oct 2015
Results rendering page with 400 table rows, and
EditRowPlugin active:
Before: # spent 4.41s (10.1ms+4.40) within Foswiki::Render::_putBackProtected which was called 45 times, avg 97.9ms/call:
After: # spent 62.9ms within Foswiki::Render::_putBackProtected which was called 45 times, avg 1.40ms/call:
--
GeorgeClark - 15 Oct 2015
Excellent optimization achieved.
Regexes used to be fast, alas they aren't anymore on unicode text.
I recently investigated a considerable slow-down upgrading from Foswiki 1.1.9 to 2.0.3. It turned out that the system's default perl 5.18.2 was the problem. Regexing unicode strings was back to normal speed on a more recent perl, in this case installing a local 5.22.1 did the trick.
But looking at the patch I get the impression that the new code is more efficient not only because of avoiding regexes.
Avoiding regexes is not something we should encourage as that's not the mindset when programming perl. However we have to face the situation of linux distributions that currently almost all ship a perl (mostly 5.18.2) that is
not able to regex unicode text flawlessly. Foswiki will inevitably be slower by a factor of 5-20 (see also
perlunicode). This also affects a lot simpler code than
putBackBlocks
.
--
MichaelDaum - 21 Jan 2016