Performance Issue

Hello

I'm quite new and unfamiliar with the foswiki site that I've inherited to look after, so please bare with me.

For several months we've been struggling with very slow performance issues.

Only a small site running 900+ topics, with:

Server specs:

- AWS server

- 2GB RAM

- 1 x 1.83 Ghz

- Windows 2012 Server Standard 64bit operating system

I've also tried out the performance considerations (http://foswiki.org/Support.PerformanceSupplement) with little effect.

Firebug results are pointing to a server response time ranging from 10s ~ 50s. I'm pretty sure I'm missing something obvious...

Any help is appreciated.

Cheers

-- AnthonyKawamoto - 09 Oct 2014

I'm not quite sure where to start. I'm not really familiar with the Amazon cloud offering, but certainly for simple pages, 10-50 seconds is very excessive. If the pages have %SEARCH macros, then response time can get much higher depending upon the search. But generally you should be able to get response times to well under 5 seconds.

• What do you get for response time on a simple page, no search, like http://foswiki.org/System/TipTopic008
• How about with plain text, like http://foswiki.org/System/TipTopic008?raw=all
• Is the response time repeatable, or if you get the same page over and over, does it vary significantly.
• What are the component times like in firebug? JQuery, CSS load times, Graphics load times, base view script time, etc.

• What web server is installed? (Apache, IIS, ?)
• Are you using client side caching (ETags, Expires times, etc. to minimize server requests)
• Which version of perl? (ActiveState, Strawberry, ?)
• Do you have FastCGI / fcgid or mod_perl installed.

-- GeorgeClark - 09 Oct 2014

The pages that were yielding the worst results all had search macros. My answers below:

• What do you get for response time on a simple page, no search, like http://foswiki.org/System/TipTopic008 - I tested on a blank topic and initially get 5.4s, then drops to 4.0~4.9s
• How about with plain text, like http://foswiki.org/System/TipTopic008?raw=all - plain text gives me the same result as above
• Is the response time repeatable, or if you get the same page over and over, does it vary significantly. - varies, but consistently above 4s. The instances of 10+s aren't as repeated; it can vary down to 4s, up to 50s
• What are the component times like in firebug? JQuery, CSS load times, Graphics load times, base view script time, etc. - on average, JS 120ms, CSS 180ms, images 250ms, HTML 4s

• What web server is installed? (Apache, IIS, ?) - Apache 2
• Are you using client side caching (ETags, Expires times, etc. to minimize server requests) - I don't think any client side caching is done (sorry, not too familiar)
• Which version of perl? (ActiveState, Strawberry, ?) - Strawberry
• Do you have FastCGI / fcgid or mod_perl installed. - Previously mod_perl. Reverted after seeing no effect

-- AnthonyKawamoto - 09 Oct 2014

I'm running Foswiki on Windows here and page loads are less than 2 seconds in general.

Which version of Foswiki are you running?

We do have FastCGI installed? That shaved as much as half a second off our page load times (this is from memory). I surprised that mod_perl did not give you similar benefits, but ...

-- JulianLevens - 09 Oct 2014

Running on Foswiki 1.1.9

I tried mod_perl again and it seems to have made a minor difference.. but not enough. I'm still seeing page loading at 15s+.

It generally seems to occur more on %SEARCH pages, but the initial log in is also taking an age to load too.

-- AnthonyKawamoto - 10 Oct 2014

Yes for some reason Windows does not perform as well as linux. I think it has something to do with the way the different filesystems work. However, on my Win 7 laptop I've tested FW 1.1.9 with Windows and Linux with very little difference between them. Whereas the company Xeon Server and Windows 2003 performs slowly.

I was hoping that Microsoft had improved something in the OS to improve filesystem performance and my Win7 experience suggested that they had. Alas, your Server 2012 experience suggests the opposite.

Options you can look into are:

• DBIStoreContrib
  • This can really help with complex queries
• NativeSearchContrib
  • One factor that impacts performance is that on Windows you are configured to use the PurePerl search algorithm. Perl has to read one topic at a time to perform regex scans to match your search criteria. On linux the grep tool does that job, but that's written in C and has much better i/o performance. That's why NativeSearchContrib may help it's effectively a replacement grep for FW also written in C. The docs for this recommend it for mod_perl use, but on Windows it's a benefit in all cases.

Note that Strawberry perl includes a full set of development tools (C complier etc), so it's quite easy to build the code in NativeSearchContrib. Of course you may not have the access to do that.

I'd love to dig into what really causes Windows to sometimes perform poorly with FW, but I do not have the time alas. I'm slowly working towards moving our FW box over to linux while upgrading FW at the same time.

I hope one of the above works for you. I'm not sure I can be much more help, but please keep us posted on how you're getting on and any solution you discover.

One final thing to consider are your templates. I had tweaked our templates to add features: such as HierarchicalNavigation. However, this is quite an expensive operation. By removing this and hard coding the WebLeftBarWebsList with our Webs saved about 2.5 seconds on every page. This change has made a big difference to making our FW feeling reasonably responsive. Of course that does not solve some slow pages with complex %SEARCHes, but it was a big help.

Good luck.

-- JulianLevens - 10 Oct 2014

Appreciate all of your help!

I see your point about limitations on Windows. I'll give DBIStoreContrib a try, if that doesn't work I'll see to migrating to Linux. Sounds like a safe bet to me...

On a side note, I doubled the RAM allocation which has drastically improved performance for non-%SEARCH topics, but little to no effect on the more complex ones. I could just keep on upping it until I get better results.. but I'd prefer to improve my FW set up.

Will keep you posted on my progress

Update: basically I moved everything to Ubuntu 14.04 / Apache2.4 with FCGI enabled. This seems to have at least halved query times. Then I threw more RAM and CPU at it which made it quite fast now