LBCache, the vBulletin cache

Introduction

If you admin a server running vBulletin, it's almost certain that you'll have had problems with performance. There's plenty of scope for server tuning, and there are even a few modules for template caching; but what happens when you've implemented all of these, and performance is still poor?

LBCache is a PHP script which implements caching of forums and threads. Installation is fairly simple, the cache is highly configurable, and - best of all - the performance boost can be huge. Real-world tests on a busy server gave a halving of CPU load (and a similar reduction in the number of MySQL queries) when the cache was running. In situations where a handful of threads are being hit hard by visitors (eg if a thread has been Slashdotted), the reduction in load will be much greater. Benchmark figures below suggest that serving pages from the cache uses around one tenth of the CPU as non-cached pages.

server load

Requirements

LBC is incompatible with vBulletin's gzip option (Admin CP -> vBulletin Options -> Cookies and HTTP Header Options) . But note that it's generally much better to implement gzip compression in Apache, rather than vBulletin anyway

Installation

Installation is fairly simple, but requires a very basic knowledge of PHP. Full instructions are included with the code; but if you don't have the technical know-how to install LBCache, an installation service is available for 25 USD

I also work as a Linux consultant. Having problems with your server? I can help. Again, contact me for further details

Purchase

LBCache is available for 50 USD - a bargain considering the hunderds of dollars you'll save on hardware upgrades. To purchase a copy, please click the paypal button below, or contact me at pete@linuxbox.co.uk


The Gory Details

Caching is always an attractive option when trying to boost performance; but for a forum, where content is constantly changing, it seems like a no-go at first glance. Not only are threads regularly being updated, each user's view of the content will be different (some users can see secret forums, some have different display preferences set, and invariably the forum will display the user's username at some place). Certainly it would be possible to cache parts of the templates which are standard to all users (the header, footer etc), but it wouldn't offer much of an improvement in performance ... especially if a template caching module is already being used.

LBCache gets around these issues by only operating on non-logged in users. Since these users all see the same view of the forum, we can cache the entire page, then serve it up next time a non-logged in user makes the same request. We still have the overhead of PHP, but we can eliminate the majority of vBulletin's database queries and template parsing.

Benchmarking

As you would expect, using PHP to simply echo out a cached page is a lot faster that vBulletin contructing the page, but how fast? The following benchmarks were made on an otherwise unused box running vBulletin 3.7.1, using ab -n 500 -c 20 http://example.com ('request the url 500 times, using 20 parallel requests')

First, without LBCache:

Server Software:        Apache/2.2.9
Document Length:        171946 bytes

Concurrency Level:      20
Time taken for tests:   87.248046 seconds
Complete requests:      500
Failed requests:        43
   (Connect: 0, Length: 43, Exceptions: 0)
Write errors:           0
Total transferred:      86241209 bytes
HTML transferred:       85972949 bytes
Requests per second:    5.73 [#/sec] (mean)
Time per request:       3489.922 [ms] (mean)
Time per request:       174.496 [ms] (mean, across all concurrent requests)
Transfer rate:          965.28 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:  2409 3478 201.1   3468    4465
Waiting:     2393 3348 186.8   3338    4348
Total:       2409 3478 201.1   3468    4465

Percentage of the requests served within a certain time (ms)
  50%   3468
  66%   3515
  75%   3551
  80%   3573
  90%   3645
  95%   3729
  98%   4127
  99%   4285
 100%   4465 (longest request)

The test took 87 seconds, averaging just over 5 requests per second.

Now lets try with LBCache enabled

Server Software:        Apache/2.2.9
Document Length:        172471 bytes

Concurrency Level:      20
Time taken for tests:   8.619330 seconds
Complete requests:      500
Failed requests:        0
Write errors:           0
Total transferred:      86354500 bytes
HTML transferred:       86235500 bytes
Requests per second:    58.01 [#/sec] (mean)
Time per request:       344.773 [ms] (mean)
Time per request:       17.239 [ms] (mean, across all concurrent requests)

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    6  27.9      0     163
Processing:    33  335 134.7    339     868
Waiting:        0  237 125.6    229     667
Total:         33  342 137.5    339     912

Percentage of the requests served within a certain time (ms)
  50%    339
  66%    393
  75%    420
  80%    449
  90%    513
  95%    558
  98%    667
  99%    734
 100%    912 (longest request)

Much better. All 500 requests were served in 8.6 seconds, with Apache averaging 58 requests per second. That's a 10 fold increase.

Don't get excited just yet though - this test represents a rather optomistic scenario in which every request is handled by the cache. In real life there'll be a lot of cache misses, and a lot of requests where the cache is ignore (because the user is logged in). Still, if a thread gets slashdotted (resulting in thousands of guests viewing it), this 10 fold increase is realistic ... and those are the times when you'll most appreciate the cache

Caching Strategies

Given that forums are meant to be interactive, we don't want to be caching pages for too long. LBCache implements caching in three parts of vbulletin:

A simple cron job is used to periodically flush expired pages, allowing you to set different expiry times for each of these three classes of page. Other pages (eg search, register etc) are never cached.

Naturally the cache period - in particular that for threads - will have an effect on performance. Too short and you'll find that most guests are missing the cache; too long and not only will guests be seeing old pages, you'll also end up with a huge cache. One compromise is to run LBCache only on the index and forum pages, setting a relatively short cache period (say 15 minutes). You'll still experience some benefits, but without the disk usage caused by thread caching. Or you may decide to only use LBCache at peak times (possibly enabling it automatically via a cron job (think sed -i)). The choice is yours.


LBCache in Action

Real-world Performance

The Apache Benchmark figures above aren't terribly realistic in day-to-day forum usage, unless a handful of threads are being hit heavily. Lets take a look at LBCache in action on a live site. The machine is a quad core AMD Phenom with 8 GB RAM, and the stats were generated by Munin. Thread caching was set to 1 hour, forum caching to 30 minutes, and the index page to 10 minutes.

Controlling the Cache

The cache is nothing more than a directory containing html files. When a client makes a request, LBCache first checks the cache. If the requested file exists, the cached version is delivered; if it doesn't, control is passed on to vBulletin as usual, and the resulting page is placed in the cache just before it is sent to the client. So, implementing a caching period is simply a case of using a cron job to periodically run find, eg:

find /var/www/html/lbcache -mmin +60 -name thread_\*|xargs rm -f

(Incidentally, the cache dir doesn't have to exist under the web tree)

In theory you could query each cached thread against the vBulletin database, and only purge those that have new posts in them (since vBulletin usefully keeps track of the last post time), but I'm not sure there's much to be gained here (plus it wouldn't detect if a post on the thread had been edited)

Logging

LBCache can be configured to log all activity to a file. With a combination of grep and wc -l, you can generate statistics on cache hits and misses, which is particularly useful when deciding on thread caching times. For instance, 

$ grep "February 17" /var/log/lbcache.log|grep THREAD_HIT|wc -l
16215
$ grep "February 17" /var/log/lbcache.log|grep THREAD_MISS|wc -l
24127

So, so far today we've had 28,860 pages that were ignored by the cache (because the user was logged in), 24,127 cases where a thread was requested but was not found in the cache, and 16,215 cases where the request waswas found in the cache. In other words, around 40% of guest thread requests were handled by the cache. With these figures in mind we can then make educated guesses as to the best thread cache period. In the above example, the thread cache period was 5 hours. Early testing with a 1 hour cache period resulted in around 10% of thread requests hitting the cache, while a 24 hour thread cache was giving a hit rate of 60%. Its goes without saying that these figures are very dependent on usage patterns on your own forum.


Feedback

Questions, thoughts, suggestions? Mail me at pete@linuxbox.co.uk

Services

Code

vBulletin

Data

Fun Stuff



linux support email pete@linuxbox.co.uk
(+44) 07890 592198