Introduction
What is Testing?
Apache Benchmark (ab) and Siege are two tools that help us to accomplish load tests. Load tests are important elements in conducting stress and performance tests. Roughly speaking...
...
For more info on the differences between these types of tests, check out http://agiletesting.blogspot.com/2005/02/performance-vs-load-vs-stress-testing.html.
Setting Benchmarks
We are primarily concerned with performance testing. At the beginning of a project, certain benchmarks should be set from conversing with a client.
What does peak traffic look like on their existing site?
What extra requests could we expect from new features? (ex: a feature that communicates with an external resource will consume resources differently than a new static page)
How fast do certain pages need to load?
How responsive do certain features need to be? (ex: do notifications need to be sent from a site immediately after an action, or can we batch them for later delivery?)
Once we've made some estimates on the performance benchmarks that we need to hit, we can design a series of load tests to see how our application behaves as it strives to meet those benchmarks.
Warnings
DO NOT perform tests on Pantheon Dev/Test servers; although Test is a rough approximation of Live (albeit with fewer resources), if you overload Test, Pantheon may shut down your Test and Dev instances.
DO NOT perform tests on a live site during peak hours.
Installing Tools
Apache Benchmark
"ab" should come pre-installed on almost any server or computer.
Siege
On your Mac, definitely use Homebrew to install Siege (brew install siege). If you haven't installed Brew, check out http://brew.sh.
On other *nix environments, use your normal package manager to handle the installation. On Kalabox, it's a simple "apt-get install siege".
Basic Anonymous Load Testing
Apache Benchmark
| Code Block |
|---|
# send 10 requests by 5 concurrent users to the url ab -n 10 -c 5 http://www.kalamuna.com/ # 500 requests by 50 concurrent users ab -n 500 -c 50 http://www.kalamuna.com/ # authenticate when blocked by .htaccess or other auth ab -n 10 -c 5 -A kala:kala http://www.kalamuna.com/ |
| Code Block |
|---|
# explicitly accept encoding for better results ab -n 100 -c 50 -k -H 'Accept-Encoding: gzip,deflate' http://www.kalamuna.com/ |
Documentation
http://httpd.apache.org/docs/2.2/programs/ab.html
Siege
| Code Block |
|---|
siege --concurrent=50 --reps=100 http://www.kalamuna.com |
Basic Authenticated Testing
Apache Benchmark
To have ab make its requests as an authenticated Drupal user, you'll need to use the "C" (cookie) flag with a valid session cookie. To do this...
Log in as a user with the proper access for the page you want to test with ab.
Open up the Chrome dev tools and look at the "Resources" tab. Copy both the key and value for the session.
Insert those values in the format below as arguments for the "C" flag (see below)
| Code Block | ||
|---|---|---|
| ||
ab -n 400 -c 10 -C 'SESSb43b2d1d084de3872c89b0b125b64564=Jafuk06rppYAXIxWaU0LY2VmqxN997DsKU3BSgfArCM' http://www.kalamuna.com/admin |
...
You can add helpful snippets of code to your template files or elsewhere to make sure requests are being sent as an authenticated user, more on that here.
Siege
| Code Block |
|---|
siege --concurrent=50 --reps=100 --header="Cookie: SESSb43b2d1d084de3872c89b0b125b64564=Jafuk06rppYAXIxWaU0LY2VmqxN997DsKU3BSgfArCM" http://www.kalamuna.com |
...
Simulating Real User Load
The above stress testing methods are great for simulating a bunch of people hitting a single page, but what about stress testing the application as a whole? Some errors don't appear unless multiple users are requesting different resources all at once. Case in point: a mysterious seg-fault error was happening on one of our sites, accompanied by a cryptic PHP Fatal error in template.inc. Users were intermittently experiencing 500 responses, but it was difficult to ascertain the pattern in the error's appearance. Turns out that a number of curl_exec calls, used to interact with an external API, were taking longer than expected to return results. For each curl_exec call, a new php-fpm worker was spawned. Since most NGINX configurations only allow a finite number of these workers, at a certain point we were hitting the limit, causing other areas of the application to fail when they needed a spare worker to process a page.
...
If you're looking to better recreate the usage pattern of real users, there are some folks who have tried creating scripts/other tools for generating URL lists. I haven't tried anything with these, but you can checkout https://drupal.org/node/48894 and report back!
Apache Benchmark Example Results
TODO: Describe how to read/use this information. Add some Siege examples.
| Code Block |
|---|
ab -n 100 -c 50 http://erinmillssoccer.com/ ============================================= Time taken for tests: 1.374944 seconds Total transferred: 7502000 bytes HTML transferred: 7453600 bytes Requests per second: 72.73 [#/sec] (mean) Time per request: 687.472 [ms] (mean) Time per request: 13.749 [ms] (mean, across all concurrent requests) Transfer rate: 5328.22 [Kbytes/sec] received ab -n 100 -c 50 -k -H 'Accept-Encoding: gzip,deflate' http://erinmillssoccer.com/ ================================================================================== Time taken for tests: 1.267539 seconds Total transferred: 1280900 bytes HTML transferred: 1230100 bytes Requests per second: 78.89 [#/sec] (mean) Time per request: 633.770 [ms] (mean) Time per request: 12.675 [ms] (mean, across all concurrent requests) Transfer rate: 986.16 [Kbytes/sec] received ********* after including deflate.conf to compress Time taken for tests: 1.503498 seconds Total transferred: 1281100 bytes HTML transferred: 1230300 bytes Requests per second: 66.51 [#/sec] (mean) Time per request: 751.749 [ms] (mean) Time per request: 15.035 [ms] (mean, across all concurrent requests) Transfer rate: 832.06 [Kbytes/sec] received |
Resources
...