gitlab-org--gitlab-foss/qa
Rémy Coutable 04df573f1b
[QA] Make sure we wait for the deploy key list to load
Signed-off-by: Rémy Coutable <remy@rymai.me>
2018-06-26 17:02:13 +02:00
..
bin Fix bootable scenario arguments for OptionParser 2017-11-10 11:11:22 +00:00
qa [QA] Make sure we wait for the deploy key list to load 2018-06-26 17:02:13 +02:00
spec Prevent ENV from leaking variables in CI output 2018-06-08 21:41:42 -07:00
.gitignore Make it possible to define global scenario attributes 2017-11-09 11:53:57 +00:00
.rspec Add GitLab QA integrations tests to GitLab CE / EE 2017-03-09 10:36:27 +01:00
Dockerfile Add gcloud and kubectl to Dockerfile for QA 2018-06-05 18:39:06 +02:00
Gemfile Implement other ssh keys and use ssh-keygen instead 2018-03-29 19:40:32 +08:00
Gemfile.lock Bump nokogiri to 1.8.2 in qa Gemfile 2018-05-01 10:57:29 -05:00
qa.rb Added Wiki Scenario Automation gitlab-qa#263 2018-06-20 17:22:05 +00:00
README.md Rename package-qa in docs 2018-03-19 10:04:23 +01:00

GitLab QA - Integration tests for GitLab

This directory contains integration tests for GitLab.

It is part of the GitLab QA project.

What is it?

GitLab QA is an integration tests suite for GitLab.

These are black-box and entirely click-driven integration tests you can run against any existing instance.

How does it work?

  1. When we release a new version of GitLab, we build a Docker images for it.
  2. Along with GitLab Docker Images we also build and publish GitLab QA images.
  3. GitLab QA project uses these images to execute integration tests.

Validating GitLab views / partials / selectors in merge requests

We recently added a new CI job that is going to be triggered for every push event in CE and EE projects. The job is called qa:selectors and it will verify coupling between page objects implemented as a part of GitLab QA and corresponding views / partials / selectors in CE / EE.

Whenever qa:selectors job fails in your merge request, you are supposed to fix page objects. You should also trigger end-to-end tests using package-and-qa manual action, to test if everything works fine.

How can I use it?

You can use GitLab QA to exercise tests on any live instance! For example, the following call would login to a local GDK instance and run all specs in qa/specs/features:

bin/qa Test::Instance http://localhost:3000

Writing tests

  1. Using page objects

Running specific tests

You can also supply specific tests to run as another parameter. For example, to run the repository-related specs, you can execute:

bin/qa Test::Instance http://localhost qa/specs/features/repository/

Since the arguments would be passed to rspec, you could use all rspec options there. For example, passing --backtrace and also line number:

bin/qa Test::Instance http://localhost qa/specs/features/login/standard_spec.rb:3 --backtrace

Overriding the authenticated user

Unless told otherwise, the QA tests will run as the default root user seeded by the GDK.

If you need to authenticate as a different user, you can provide the GITLAB_USERNAME and GITLAB_PASSWORD environment variables:

GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password bin/qa Test::Instance https://gitlab.example.com

If your user doesn't have permission to default sandbox group gitlab-qa-sandbox, you could also use another sandbox group by giving GITLAB_SANDBOX_NAME:

GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password GITLAB_SANDBOX_NAME=jsmith-qa-sandbox bin/qa Test::Instance https://gitlab.example.com

In addition, the GITLAB_USER_TYPE can be set to "ldap" to sign in as an LDAP user:

GITLAB_USER_TYPE=ldap GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password GITLAB_SANDBOX_NAME=jsmith-qa-sandbox bin/qa Test::Instance https://gitlab.example.com

All supported environment variables are here.

Building a Docker image to test

Once you have made changes to the CE/EE repositories, you may want to build a Docker image to test locally instead of waiting for the gitlab-ce-qa or gitlab-ee-qa nightly builds. To do that, you can run from this directory:

docker build -t gitlab/gitlab-ce-qa:nightly .