5.7 KiB
Multi-project pipelines [PREMIUM]
When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project inter-dependencies.
Overview
GitLab CI/CD is a powerful continuous integration tool that works not only per project, but also across projects. When you configure GitLab CI for your project, you can visualize the stages of your jobs on a pipeline graph.
In the Merge Request Widget, multi-project pipeline mini-graphs are displayed, and when hovering or tapping (on touchscreen devices) they will expand and be shown adjacent to each other.
Multi-project pipelines are useful for larger products that require cross-project inter-dependencies, such as those adopting a microservices architecture.
Use cases
Let's assume you deploy your web app from different projects in GitLab:
- One for the free version, which has its own pipeline that builds and tests your app
- One for the paid version add-ons, which also pass through builds and tests
- One for the documentation, which also builds, tests, and deploys with an SSG
With Multi-Project Pipelines, you can visualize the entire pipeline, including all stages of builds and tests for the three projects.
Triggering multi-project pipelines through API
When you use the CI_JOB_TOKEN to trigger pipelines, GitLab
recognizes the source of the job token, and thus internally ties these pipelines
together, allowing you to visualize their relationships on pipeline graphs.
These relationships are displayed in the pipeline graph by showing inbound and outbound connections for upstream and downstream pipeline dependencies.
Creating multi-project pipelines from .gitlab-ci.yml
Introduced in GitLab Premium 11.8.
Triggering a downstream pipeline using a bridge job
Before GitLab 11.8, it was necessary to implement a pipeline job that was responsible for making the API request to trigger a pipeline in a different project.
In GitLab 11.8, GitLab provides a new CI/CD configuration syntax to make this task easier, and avoid needing GitLab Runner for triggering cross-project pipelines. The following illustrates configuring a bridge job:
rspec:
stage: test
script: bundle exec rspec
staging:
variables:
ENVIRONMENT: staging
stage: deploy
trigger: my/deployment
In the example above, as soon as rspec job succeeds in the test stage,
the staging bridge job is going to be started. The initial status of this
job will be pending. GitLab will create a downstream pipeline in the
my/deployment project and, as soon as the pipeline gets created, the
staging job will succeed. my/deployment is a full path to that project.
The user that created the upstream pipeline needs to have access rights to the
downstream project (my/deployment in this case). If a downstream project can
not be found, or a user does not have access rights to create pipeline there,
the staging job is going to be marked as failed.
CAUTION: Caution:
staging will succeed as soon as a downstream pipeline gets created.
GitLab does not support status attribution yet, however adding first-class
trigger configuration syntax is ground work for implementing
status attribution.
NOTE: Note:
Bridge jobs do not support every configuration entry that a user can use
in the case of regular jobs. Bridge jobs will not to be picked by a Runner,
thus there is no point in adding support for script, for example. If a user
tries to use unsupported configuration syntax, YAML validation will fail upon
pipeline creation.
Specifying a downstream pipeline branch
It is possible to specify a branch name that a downstream pipeline will use:
rspec:
stage: test
script: bundle exec rspec
staging:
stage: deploy
trigger:
project: my/deployment
branch: stable-11-2
Use a project keyword to specify full path to a downstream project. Use
a branch keyword to specify a branch name.
GitLab will use a commit that is currently on the HEAD of the branch when creating a downstream pipeline.
Passing variables to a downstream pipeline
Sometimes you might want to pass variables to a downstream pipeline.
You can do that using the variables keyword, just like you would when
defining a regular job.
rspec:
stage: test
script: bundle exec rspec
staging:
variables:
ENVIRONMENT: staging
stage: deploy
trigger: my/deployment
The ENVIRONMENT variable will be passed to every job defined in a downstream
pipeline. It will be available as an environment variable when GitLab Runner picks a job.
Limitations
Because bridge jobs are a little different to regular jobs, it is not possible to use exactly the same configuration syntax here, as one would normally do when defining a regular job that will be picked by a runner.
Some features are not implemented yet. For example, support for environments.
Configuration keywords available for bridge jobs are:
trigger(to define a downstream pipeline trigger)stageallow_failureonlyandexceptwhenextends