Merge branch 'docs/qa/gb/document-integration-tests-in-merge-requests' into 'master'
Add docs about end-to-end testing / GitLab QA tests Closes #39637 See merge request gitlab-org/gitlab-ce!16043
This commit is contained in:
commit
780cc843df
|
@ -0,0 +1,80 @@
|
||||||
|
# End-to-End Testing
|
||||||
|
|
||||||
|
## What is End-to-End testing?
|
||||||
|
|
||||||
|
End-to-End testing is a strategy used to check whether your application works
|
||||||
|
as expected across entire software stack and architecture, including
|
||||||
|
integration of all microservices and components that are supposed to work
|
||||||
|
together.
|
||||||
|
|
||||||
|
## How do we test GitLab?
|
||||||
|
|
||||||
|
We use [Omnibus GitLab][omnibus-gitlab] to build GitLab packages and then we
|
||||||
|
test these packages using [GitLab QA][gitlab-qa] project, which is entirely
|
||||||
|
black-box, click-driven testing framework.
|
||||||
|
|
||||||
|
### Testing nightly builds
|
||||||
|
|
||||||
|
We run scheduled pipeline each night to test nightly builds created by Omnibus.
|
||||||
|
You can find these nightly pipelines at [GitLab QA pipelines page][gitlab-qa-pipelines].
|
||||||
|
|
||||||
|
### Testing code in merge requests
|
||||||
|
|
||||||
|
It is possible to run end-to-end tests (eventually being run within a
|
||||||
|
[GitLab QA pipeline][gitlab-qa-pipelines]) for a merge request by triggering
|
||||||
|
the `package-qa` manual action, that should be present in a merge request
|
||||||
|
widget.
|
||||||
|
|
||||||
|
Mmanual action that starts end-to-end tests is also available in merge requests
|
||||||
|
in Omnibus GitLab project.
|
||||||
|
|
||||||
|
Below you can read more about how to use it and how does it work.
|
||||||
|
|
||||||
|
#### How does it work?
|
||||||
|
|
||||||
|
Currently, we are using _multi-project pipeline_-like approach to run QA
|
||||||
|
pipelines.
|
||||||
|
|
||||||
|
1. Developer triggers a manual action, that can be found in CE and EE merge
|
||||||
|
requests. This starts a chain of pipelines in multiple projects.
|
||||||
|
|
||||||
|
1. The script being executed triggers a pipeline in GitLab Omnibus and waits
|
||||||
|
for the resulting status. We call this a _status attribution_.
|
||||||
|
|
||||||
|
1. GitLab packages are being built in Omnibus pipeline. Packages are going to be
|
||||||
|
pushed to Container Registry.
|
||||||
|
|
||||||
|
1. When packages are ready, and available in the registry, a final step in the
|
||||||
|
pipeline, that is now running in Omnibus, triggers a new pipeline in the GitLab
|
||||||
|
QA project. It also waits for a resulting status.
|
||||||
|
|
||||||
|
1. GitLab QA pulls images from the registry, spins-up containers and runs tests
|
||||||
|
against a test environment that has been just orchestrated by the `gitlab-qa`
|
||||||
|
tool.
|
||||||
|
|
||||||
|
1. The result of the GitLab QA pipeline is being propagated upstream, through
|
||||||
|
Omnibus, back to CE / EE merge request.
|
||||||
|
|
||||||
|
#### How do I write tests?
|
||||||
|
|
||||||
|
In order to write new tests, you first need to learn more about GitLab QA
|
||||||
|
architecture. See the [documentation about it][gitlab-qa-architecture] in
|
||||||
|
GitLab QA project.
|
||||||
|
|
||||||
|
Once you decided where to put test environment orchestration scenarios and
|
||||||
|
instance specs, take a look at the [relevant documentation][instance-qa-readme]
|
||||||
|
and examples in [the `qa/` directory][instance-qa-examples].
|
||||||
|
|
||||||
|
## Where can I ask for help?
|
||||||
|
|
||||||
|
You can ask question in the `#qa` channel on Slack (GitLab internal) or you can
|
||||||
|
find an issue you would like to work on in [the issue tracker][gitlab-qa-issues]
|
||||||
|
and start a new discussion there.
|
||||||
|
|
||||||
|
[omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab
|
||||||
|
[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa
|
||||||
|
[gitlab-qa-pipelines]: https://gitlab.com/gitlab-org/gitlab-qa/pipelines
|
||||||
|
[gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md
|
||||||
|
[gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues
|
||||||
|
[instance-qa-readme]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/README.md
|
||||||
|
[instance-qa-examples]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa
|
|
@ -65,6 +65,13 @@ Everything you should know about how to test Rake tasks.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## [End-to-end tests](end_to_end_tests.md)
|
||||||
|
|
||||||
|
Everything you should know about how to run end-to-end tests using
|
||||||
|
[GitLab QA][gitlab-qa] testing framework.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Spinach (feature) tests
|
## Spinach (feature) tests
|
||||||
|
|
||||||
GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426)
|
GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426)
|
||||||
|
@ -89,3 +96,4 @@ test should be re-implemented using RSpec instead.
|
||||||
[Capybara]: https://github.com/teamcapybara/capybara
|
[Capybara]: https://github.com/teamcapybara/capybara
|
||||||
[Karma]: http://karma-runner.github.io/
|
[Karma]: http://karma-runner.github.io/
|
||||||
[Jasmine]: https://jasmine.github.io/
|
[Jasmine]: https://jasmine.github.io/
|
||||||
|
[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa
|
||||||
|
|
|
@ -121,6 +121,9 @@ running feature tests (i.e. using Capybara) against it.
|
||||||
The actual test scenarios and steps are [part of GitLab Rails] so that they're
|
The actual test scenarios and steps are [part of GitLab Rails] so that they're
|
||||||
always in-sync with the codebase.
|
always in-sync with the codebase.
|
||||||
|
|
||||||
|
Read a separate document about [end-to-end tests](end_to_end_tests.md) to
|
||||||
|
learn more.
|
||||||
|
|
||||||
[multiple pieces]: ../architecture.md#components
|
[multiple pieces]: ../architecture.md#components
|
||||||
[GitLab Shell]: https://gitlab.com/gitlab-org/gitlab-shell
|
[GitLab Shell]: https://gitlab.com/gitlab-org/gitlab-shell
|
||||||
[GitLab Workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse
|
[GitLab Workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse
|
||||||
|
|
Loading…
Reference in New Issue