Documentation for JUnit XML Test Summary In MR widget
This commit is contained in:
parent
218fc3e3db
commit
6e9e61dc32
|
@ -133,6 +133,7 @@ scales to run your tests faster.
|
|||
- [GitLab CI/CD](ci/README.md): Explore the features and capabilities of Continuous Integration, Continuous Delivery, and Continuous Deployment with GitLab.
|
||||
- [Review Apps](ci/review_apps/index.md): Preview changes to your app right from a merge request.
|
||||
- [Pipeline Graphs](ci/pipelines.md#pipeline-graphs)
|
||||
- [JUnit test reports](ci/junit_test_reports.md)
|
||||
|
||||
### Package
|
||||
|
||||
|
|
|
@ -43,6 +43,10 @@ There's also a collection of repositories with [example projects](https://gitlab
|
|||
- [Using `dpl` as deployment tool](deployment/README.md)
|
||||
- [The `.gitlab-ci.yml` file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml)
|
||||
|
||||
## Test Reports
|
||||
|
||||
[Collect test reports in Verify stage](../junit_test_reports.md).
|
||||
|
||||
## Code Quality analysis
|
||||
|
||||
**(Starter)** [Analyze your project's Code Quality](code_quality.md).
|
||||
|
|
Binary file not shown.
After Width: | Height: | Size: 9.3 KiB |
|
@ -0,0 +1,102 @@
|
|||
# JUnit test reports
|
||||
|
||||
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/45318) in GitLab 11.2.
|
||||
Requires GitLab Runner 11.2 and above.
|
||||
|
||||
## Overview
|
||||
|
||||
It is very common that a [CI/CD pipeline](pipelines.md) contains a
|
||||
test job that will verify your code.
|
||||
If the tests fail, the pipeline fails and users get notified. The person that
|
||||
works on the merge request will have to check the job logs and see where the
|
||||
tests failed so that they can fix them.
|
||||
|
||||
You can configure your job to use JUnit test reports, and GitLab will display a
|
||||
report on the merge request so that it's easier and faster to identify the
|
||||
failure without having to check the entire log.
|
||||
|
||||
## Use cases
|
||||
|
||||
Consider the following workflow:
|
||||
|
||||
1. Your `master` branch is rock solid, your project is using GitLab CI/CD and
|
||||
your pipelines indicate that there isn't anything broken.
|
||||
1. Someone from you team submits a merge request, a test fails and the pipeline
|
||||
gets the known red icon. To investigate more, you have to go through the job
|
||||
logs to figure out the cause of the failed test, which usually contain
|
||||
thousands of lines.
|
||||
1. You configure the JUnit test reports and immediately GitLab collects and
|
||||
exposes them in the merge request. No more searching in the job logs.
|
||||
1. Your development and debugging workflow becomes easier, faster and efficient.
|
||||
|
||||
## How it works
|
||||
|
||||
First, GitLab Runner uploads all JUnit XML files as artifacts to GitLab. Then,
|
||||
when you visit a merge request, GitLab starts comparing the head and base branch's
|
||||
JUnit test reports, where:
|
||||
|
||||
- The base branch is the target branch (usually `master`).
|
||||
- The head branch is the source branch (the latest pipeline in each merge request).
|
||||
|
||||
The reports panel has a summary showing how many tests failed and how many were fixed.
|
||||
If no comparison can be done because data for the base branch is not available,
|
||||
the panel will just show the list of failed tests for head.
|
||||
|
||||
There are three types of results:
|
||||
|
||||
1. **Newly failed tests:** Test cases which passed on base branch and failed on head branch
|
||||
1. **Existing failures:** Test cases which failed on base branch and failed on head branch
|
||||
1. **Resolved failures:** Test cases which failed on base branch and passed on head branch
|
||||
|
||||
Each entry in the panel will show the test name and its type from the list
|
||||
above. Clicking on the test name will open a modal window with details of its
|
||||
execution time and the error output.
|
||||
|
||||
![Test Reports Widget](img/junit_test_report.png)
|
||||
|
||||
## How to set it up
|
||||
|
||||
NOTE: **Note:**
|
||||
For a list of supported languages on JUnit tests, check the
|
||||
[Wikipedia article](https://en.wikipedia.org/wiki/JUnit#Ports).
|
||||
|
||||
To enable the JUnit reports in merge requests, you need to add
|
||||
[`artifacts:reports:junit`](yaml/README.md#artifacts-reports-junit)
|
||||
in `.gitlab-ci.yml`, and specify the path(s) of the generated test reports.
|
||||
|
||||
In the following examples, the job in the `test` stage runs and GitLab
|
||||
collects the JUnit test report from each job. After each job is executed, the
|
||||
XML reports are stored in GitLab as artifacts and their results are shown in the
|
||||
merge request widget.
|
||||
|
||||
### Ruby example
|
||||
|
||||
Use the following job in `.gitlab-ci.yml`:
|
||||
|
||||
```yaml
|
||||
## Use https://github.com/sj26/rspec_junit_formatter to generate a JUnit report with rspec
|
||||
ruby:
|
||||
stage: test
|
||||
script:
|
||||
- bundle install
|
||||
- rspec spec/lib/ --format RspecJunitFormatter --out rspec.xml
|
||||
artifacts:
|
||||
reports:
|
||||
junit: rspec.xml
|
||||
```
|
||||
|
||||
### Go example
|
||||
|
||||
Use the following job in `.gitlab-ci.yml`:
|
||||
|
||||
```yaml
|
||||
## Use https://github.com/jstemmer/go-junit-report to generate a JUnit report with go
|
||||
golang:
|
||||
stage: test
|
||||
script:
|
||||
- go get -u github.com/jstemmer/go-junit-report
|
||||
- go test -v 2>&1 | go-junit-report > report.xml
|
||||
artifacts:
|
||||
reports:
|
||||
junit: report.xml
|
||||
```
|
|
@ -1092,6 +1092,52 @@ job:
|
|||
expire_in: 1 week
|
||||
```
|
||||
|
||||
### `artifacts:reports`
|
||||
|
||||
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20390) in
|
||||
GitLab 11.2. Requires GitLab Runner 11.2 and above.
|
||||
|
||||
The `reports` keyword is used for collecting test reports from jobs and
|
||||
exposing them in GitLab's UI (merge requests, pipeline views). Read how to use
|
||||
this with [JUnit reports](#artifacts-reports-junit).
|
||||
|
||||
NOTE: **Note:**
|
||||
The test reports are collected regardless of the job results (success or failure).
|
||||
You can use [`artifacts:expire_in`](#artifacts-expire_in) to set up an expiration
|
||||
date for their artifacts.
|
||||
|
||||
#### `artifacts:reports:junit`
|
||||
|
||||
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20390) in
|
||||
GitLab 11.2. Requires GitLab Runner 11.2 and above.
|
||||
|
||||
The `junit` report collects [JUnit XML files](https://www.ibm.com/support/knowledgecenter/en/SSQ2R2_14.1.0/com.ibm.rsar.analysis.codereview.cobol.doc/topics/cac_useresults_junit.html)
|
||||
as artifacts. Although JUnit was originally developed in Java, there are many
|
||||
[third party ports](https://en.wikipedia.org/wiki/JUnit#Ports) for other
|
||||
languages like Javascript, Python, Ruby, etc.
|
||||
|
||||
Below is an example of collecting a JUnit XML file from Ruby's RSpec test tool:
|
||||
|
||||
```yaml
|
||||
rspec:
|
||||
stage: test
|
||||
script:
|
||||
- bundle install
|
||||
- rspec --format RspecJunitFormatter --out rspec.xml
|
||||
artifacts:
|
||||
reports:
|
||||
junit: rspec.xml
|
||||
```
|
||||
|
||||
The collected JUnit reports will be uploaded to GitLab as an artifact and will
|
||||
be automatically [shown in merge requests](../junit_test_reports.md).
|
||||
|
||||
NOTE: **Note:**
|
||||
In case the JUnit tool you use exports to multiple XML files, you can specify
|
||||
multiple test report paths within a single job
|
||||
(`junit: [rspec-1.xml, rspec-2.xml, rspec-3.xml]`) and they will be automatically
|
||||
concatenated into a single file.
|
||||
|
||||
## `dependencies`
|
||||
|
||||
> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
|
||||
|
|
|
@ -43,8 +43,7 @@ A. Consider you are a software developer working in a team:
|
|||
|
||||
1. You checkout a new branch, and submit your changes through a merge request
|
||||
1. You gather feedback from your team
|
||||
1. You work on the implementation optimizing code with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html) **[STARTER]**
|
||||
1. You build and test your changes with GitLab CI/CD
|
||||
1. You verify your changes with [JUnit test reports](../../../ci/junit_test_reports.md) in GitLab CI/CD
|
||||
1. You request the approval from your manager
|
||||
1. Your manager pushes a commit with his final review, [approves the merge request](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html), and set it to [merge when pipeline succeeds](#merge-when-pipeline-succeeds) (Merge Request Approvals are available in GitLab Starter)
|
||||
1. Your changes get deployed to production with [manual actions](../../../ci/yaml/README.md#manual-actions) for GitLab CI/CD
|
||||
|
|
Loading…
Reference in New Issue