4.7 KiB
Testing Standards and Style Guidelines
This guide outlines standards and best practices for automated testing of GitLab CE and EE.
It is meant to be an extension of the thoughtbot testing styleguide. If this guide defines a rule that contradicts the thoughtbot guide, this guide takes precedence. Some guidelines may be repeated verbatim to stress their importance.
Factories
GitLab uses factory_girl as a test fixture replacement.
- Factory definitions live in
spec/factories/
, named using the pluralization of their corresponding model (User
factories are defined inusers.rb
). - There should be only one top-level factory definition per file.
- Make use of Traits to clean up definitions and usages.
- When defining a factory, don't define attributes that are not required for the resulting record to pass validation.
- When instantiating from a factory, don't supply extraneous attributes that aren't required by the test.
JavaScript
GitLab uses Teaspoon to run its Jasmine JavaScript specs. They can be run on
the command line via bundle exec teaspoon
, or via a web browser at
http://localhost:3000/teaspoon
when the Rails server is running.
-
JavaScript tests live in
spec/javascripts/
, matching the folder structure ofapp/assets/javascripts/
:app/assets/javascripts/behaviors/autosize.js.coffee
has a correspondingspec/javascripts/behaviors/autosize_spec.js.coffee
file. -
Haml fixtures required for JavaScript tests live in
spec/javascripts/fixtures
. They should contain the bare minimum amount of markup necessary for the test.Warning: Keep in mind that a Rails view may change and invalidate your test, but everything will still pass because your fixture doesn't reflect the latest view.
-
Keep in mind that in a CI environment, these tests are run in a headless browser and you will not have access to certain APIs, such as
Notification
, which will have to be stubbed.
RSpec
General Guidelines
- Use a single, top-level
describe ClassName
block. - Use
described_class
instead of repeating the class name being described. - Use
.method
to describe class methods and#method
to describe instance methods. - Use
context
to test branching logic. - Don't
describe
symbols (see Gotchas). - Prefer
not_to
toto_not
. - Try to match the ordering of tests to the ordering within the class.
- Try to follow the Four-Phase Test pattern, using newlines to separate phases.
Test speed
GitLab has a massive test suite that, without parallelization, can take more than an hour to run. It's important that we make an effort to write tests that are accurate and effective as well as fast.
Here are some things to keep in mind regarding test performance:
double
andspy
are faster thanFactoryGirl.build(...)
FactoryGirl.build(...)
and.build_stubbed
are faster than.create
.- Don't
create
an object whenbuild
,build_stubbed
,attributes_for
,spy
, ordouble
will do. Database persistence is slow! - Use
create(:empty_project)
instead ofcreate(:project)
when you don't need the underlying repository. Filesystem operations are slow! - Don't mark a feature as requiring JavaScript (through
@javascript
in Spinach orjs: true
in RSpec) unless it's actually required for the test to be valid. Headless browser testing is slow!
Features / Integration
- Feature specs live in
spec/features/
and should be namedROLE_ACTION_spec.rb
, such asuser_changes_password_spec.rb
. - Use only one
feature
block per feature spec file. - Use scenario titles that describe the success and failure paths.
- Avoid scenario titles that add no information, such as "successfully."
- Avoid scenario titles that repeat the feature title.
Spinach (feature) tests
GitLab moved from Cucumber to Spinach for its feature/integration tests in September 2012.
As of March 2016, we are trying to avoid adding new Spinach tests going forward, opting for RSpec feature specs.
Adding new Spinach scenarios is acceptable only if the new scenario requires
no more than one new step
definition. If more than that is required, the
test should be re-implemented using RSpec instead.