Item1382: Small speed optimisation of search code

Meta::putKeyed uses a surprising amount of cpu (using NYTProf) looping through 'all' the items in a 'type'

(I was profiling AdminToolsCategory)

by replacing this loop with a set of hash indexes, I shaved off ~4seconds from the 11-12second run time - NYTProf slows thigs down

I'm going to commit it to trunk today, and see about release branch tomorrow.

-- SvenDowideit - 27 Mar 2009

I have a suspicion that its due to the DocumentGraphics topic, which someone has added the attachment META to for all ~260 images

and so there's another issue - as it appears that topic might be loaded and parsed more than once., which i'll poke tomorrow.

-- SvenDowideit - 27 Mar 2009

Interesting observation

We have to be careful not to break something in release branch but the observation that DocumentGraphics attachment meta can slow down things is very interesting and worth investigating. I will try and benchmark the difference with and without. There is no functional need for the icons to be formally attached.

And people can still see the images in the topic as long as they are in the pub dir

-- KennethLavrsen - 27 Mar 2009

well. I backed out the INDEX hash, as it causes unit test failures, and I get the feeling that the {_loadedRev} change has more real effects.

Looking around, there seem to be alot of ways to get Foswiki to re-re-reload files - using YuiMenuContrib (which has one tmpl file containing one TMPL:DEF seems to cause Foswiki to load 165 tmpl files (lots of reloads) vs 36! in just pattern.

so there's quite alot of code analysis that should be done - in many cases a print STDERR wherever there's an open(), and in others NTYProf.

I'm going to leave it for a little, as i really need to be elsewhere

-- SvenDowideit - 28 Mar 2009

Interesting enough that I picked this up and did it. All tests pass. Sven, take a look at what I did; I didn't use your code, exactly, but I think I followed your intent accurately. If I did then set this "Waiting for Release".

-- CrawfordCurrie - 23 May 2009

i recon it looks good

-- SvenDowideit - 24 May 2009