This guide covers the various ways of performance testing a Ruby on Rails application. By referring to this guide, you will be able to:
* Understand the various types of benchmarking and profiling metrics
* Generate performance and benchmarking tests
* Use a GC-patched Ruby binary to measure memory usage and object allocation
* Understand the benchmarking information provided by Rails inside the log files
* Learn about various tools facilitating benchmarking and profiling
Performance testing is an integral part of the development cycle. It is very important that you don't make your end users wait for too long before the page is completely loaded. Ensuring a pleasant browsing experience for end users and cutting the cost of unnecessary hardware is important for any non-trivial web application.
endprologue.
h3. Performance Test Cases
Rails performance tests are a special type of integration tests, designed for benchmarking and profiling the test code. With performance tests, you can determine where your application's memory or speed problems are coming from, and get a more in-depth picture of those problems.
In a freshly generated Rails application, +test/performance/browsing_test.rb+ contains an example of a performance test:
Even though the performance tests are integration tests and hence closer to the request/response cycle by nature, you can still performance test pure model code.
Benchmarking helps find out how fast each performance test runs. Each test case is run +4 times+ in benchmarking mode.
To run performance tests in benchmarking mode:
<shell>
$ rake test:benchmark
</shell>
h5. Profiling
Profiling helps you see the details of a performance test and provide an in-depth picture of the slow and memory hungry parts. Each test case is run +1 time+ in profiling mode.
To run performance tests in profiling mode:
<shell>
$ rake test:profile
</shell>
h4. Metrics
Benchmarking and profiling run performance tests in various modes described below.
Process time measures the time taken by the process. It is unaffected by any other processes running concurrently on the same system. Hence, process time is likely to be constant for any given performance test, irrespective of the machine load.
Performance test results are also appended to +.csv+ files inside +tmp/performance+. For example, running the default +BrowsingTest#test_homepage+ will generate following five files:
As the results are appended to these files each time the performance tests are run in benchmarking mode, you can collect data over a period of time. This can be very helpful in analyzing the effects of code changes.
Sample output of +BrowsingTest#test_homepage_wall_time.csv+:
This is a very basic form of output in profiling mode:
<shell>
BrowsingTest#test_homepage (58 ms warmup)
process_time: 63 ms
memory: 832.13 KB
objects: 7882
</shell>
h6. Flat
Flat output shows the total amount of time spent in each method. "Check ruby prof documentation for a better explanation":http://ruby-prof.rubyforge.org/files/examples/flat_txt.html.
h6. Graph
Graph output shows how long each method takes to run, which methods call it and which methods it calls. "Check ruby prof documentation for a better explanation":http://ruby-prof.rubyforge.org/files/examples/graph_txt.html.
To get the best from Rails performance tests, you need to build a special Ruby binary with some super powers - "GC patch":http://rubyforge.org/tracker/download.php/1814/7062/17676/3291/ruby186gc.patch for measuring GC Runs/Time and memory/object allocation.
The process is fairly straightforward. If you've never compiled a Ruby binary before, follow these steps to build a ruby binary inside your home directory:
The following will install ruby in your home directory's +/rubygc+ directory. Make sure to replace +<homedir>+ with a full patch to your actual home directory.
Download "Rubygems":http://rubyforge.org/projects/rubygems and install it from source. Rubygem's README file should have necessary installation instructions.
And you're ready to go. Don't forget to use +gcruby+ and +gcrake+ aliases when running the performance tests.
h3. Command Line Tools
Writing performance test cases could be an overkill when you are looking for one time tests. Rails ships with two command line tools that enable quick and dirty performance testing:
Rails provides various helper methods inside Active Record, Action Controller and Action View to measure the time taken by a given piece of code. The method is called +benchmark()+ in all the three components.
Similarly, you could use this helper method inside "controllers":http://api.rubyonrails.org/classes/ActionController/Benchmarking/ClassMethods.html#M000715
This data is fairly straightforward to understand. Rails uses millisecond(ms) as the metric to measure the time taken. The complete request spent 5 ms inside Rails, out of which 2 ms were spent rendering views and none was spent communication with the database. It's safe to assume that the remaining 3 ms were spent inside the controller.
Michael Koziarski has an "interesting blog post":http://www.therailsway.com/2009/1/6/requests-per-second explaining the importance of using milliseconds as the metric.
