Item10747: Slow initial save on new topics
Priority: Normal
Current State: Confirmed
Released In: n/a
Target Release: n/a
On initial save of a new topic, there is a huge performance impact when using this plugin.
Needs to be investigated.
--
OliverKrueger - 15 May 2011
IIRC it does a search of all webs to ensure the newly generated permid is unique.
Certainly with bruteforce & forking it times out on our wiki.
With
MongoDBPlugin it works fine.
IIRC the permid is generated from topic name + timestamp; maybe we can have a configure option to avoid the uniqueness validation (or get a better hashing algorithm so we can have more faith there won't be collisions)
--
PaulHarvey - 16 May 2011
Right now, there's a %SEARCH involved in
every rest/PermLinkPlugin/view
over all topics in all of the wiki...see
get_topic_for_id()
...
getListOfWebs()
...
%SEARCH
...
So how about using dbi to store ids and to retrieve the real topic for a permalink. That'll be a nicer option for anybody else not using
MongoDBPlugin. And it would not only speed up
save
but also every
view
of a permalink.
IMHO, permalinks - md5 as well as human readable ones - should be a core feature.
--
MichaelDaum - 16 May 2011
FWIW, this change brings the request times from ~2.2s on our VM to 1.6s:
diff --git a/lib/Foswiki/Plugins/PermLinkPlugin.pm b/lib/Foswiki/Plugins/PermLinkPlugin.pm
index 40d3adf..b10d4c2 100644
--- a/lib/Foswiki/Plugins/PermLinkPlugin.pm
+++ b/lib/Foswiki/Plugins/PermLinkPlugin.pm
@@ -76,20 +78,16 @@ sub get_topic_for_md5 {
# a noobs implementation
sub get_topic_for_id {
( my $method, my $id ) = @_;
-
- my $webs = join( q{,}, Foswiki::Func::getListOfWebs() );
- my $tml = "%SEARCH{ ";
- $tml .=
- "search=\"preferences[name='" . $method . "' AND value='" . $id . "']\" ";
- $tml .= 'type="query" ';
- $tml .= 'web="' . $webs . '" ';
- $tml .= 'format="$web.$topic" ';
- $tml .= 'seperator="," ';
- $tml .= 'nonoise="on" ';
- $tml .= 'limit="1" ';
- $tml .= "}%";
-
- my $topic = Foswiki::Func::expandCommonVariables($tml);
+ my $iterator = Foswiki::Func::query(
+ "preferences[name='$method' AND value='$id']",
+ undef,
+ {
+ web => 'all',
+ recurse => 'on',
+ limit => '1'
+ }
+ );
+ my $topic = join( '.', $iterator->next() );
return $topic if Foswiki::Func::topicExists( q{}, $topic );
return q{};
--
PaulHarvey - 16 May 2011
This is faster because
MongoDBPlugin (and other query algo's) can roll this up into a single query instead of one query for each separate web.
Still, 1.6s is an eternity in the "Web 3.0" world for such a simple lookup task... here's hoping
Item10461 will see an improvement on this.
Anyway, we should raise a separate task for the job of making a more efficient lookup implementation (I'd hope it's configurable so that I don't have to run a separate DB when I'm already using
MongoDBPlugin).
The issue on this task is to figure out how to improve the save times when minting a new permid.
--
PaulHarvey - 16 May 2011