10 KiB

Raw Blame History

stage	group	info	type
Verify	Pipeline Authoring	To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments	reference

Multi-project pipelines (FREE)

Moved to GitLab Free in 12.8.

You can set up GitLab CI/CD across multiple projects, so that a pipeline in one project can trigger a downstream pipeline in another project. You can visualize the entire pipeline in one place, including all cross-project interdependencies.

For example, you might deploy your web application from three different projects in GitLab. Each project has its own build, test, and deploy process. With multi-project pipelines you can visualize the entire pipeline, including all build and test stages for all three projects.

For an overview, see the Multi-project pipelines demo.

Multi-project pipelines are also useful for larger products that require cross-project interdependencies, like those with a microservices architecture. Learn more in the Cross-project Pipeline Triggering and Visualization demo at GitLab@learn, in the Continuous Integration section.

If you trigger a pipeline in a downstream private project, on the upstream project's pipelines page, you can view:

The name of the project.
The status of the pipeline.

If you have a public project that can trigger downstream pipelines in a private project, make sure there are no confidentiality problems.

Create multi-project pipelines

To create multi-project pipelines, you can:

Define them in your .gitlab-ci.yml file.
Use the API.

Define multi-project pipelines in your `.gitlab-ci.yml` file

Moved to GitLab Free in 12.8.

When you use the trigger keyword to create a multi-project pipeline in your .gitlab-ci.yml file, you create what is called a trigger job. For example:

rspec:
  stage: test
  script: bundle exec rspec

staging:
  variables:
    ENVIRONMENT: staging
  stage: deploy
  trigger: my/deployment

In this example, after the rspec job succeeds in the test stage, the staging trigger job starts. The initial status of this job is pending.

GitLab then creates a downstream pipeline in the my/deployment project and, as soon as the pipeline is created, the staging job succeeds. The full path to the project is my/deployment.

You can view the status for the pipeline, or you can display the downstream pipeline's status instead.

The user that creates the upstream pipeline must be able to create pipelines in the downstream project (my/deployment) too. If the downstream project is not found, or the user does not have permission to create a pipeline there, the staging job is marked as failed.

Trigger job configuration limitations

Trigger jobs can use only a limited set of the GitLab CI/CD configuration keywords. The keywords available for use in trigger jobs are:

trigger
stage
allow_failure
rules
only and except
when (only with a value of on_success, on_failure, or always)
extends
needs, but not needs:project

Trigger jobs cannot use job-level persisted variables.

Specify a downstream pipeline branch

You can specify a branch name for the downstream pipeline to use. GitLab uses the commit on the head of the branch to create the downstream pipeline.

rspec:
  stage: test
  script: bundle exec rspec

staging:
  stage: deploy
  trigger:
    project: my/deployment
    branch: stable-11-2

Use:

The project keyword to specify the full path to a downstream project. In GitLab 15.3 and later, variable expansion is supported.
The branch keyword to specify the name of a branch in the project specified by project. In GitLab 12.4 and later, variable expansion is supported.

Pipelines triggered on a protected branch in a downstream project use the role of the user that ran the trigger job in the upstream project. If the user does not have permission to run CI/CD pipelines against the protected branch, the pipeline fails. See pipeline security for protected branches.

Pass artifacts to a downstream pipeline

You can pass artifacts to a downstream pipeline by using needs:project.

In a job in the upstream pipeline, save the artifacts using the artifacts keyword.

Trigger the downstream pipeline with a trigger job:

build_artifacts:
  stage: build
  script:
    - echo "This is a test artifact!" >> artifact.txt
  artifacts:
    paths:
      - artifact.txt

deploy:
  stage: deploy
  trigger: my/downstream_project

In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline by using needs:project. Set job to the job in the upstream pipeline to fetch artifacts from, ref to the branch, and artifacts: true.

test:
  stage: test
  script:
    - cat artifact.txt
  needs:
    - project: my/upstream_project
      job: build_artifacts
      ref: main
      artifacts: true

Pass artifacts to a downstream pipeline from a Merge Request pipeline

When you use needs:project to pass artifacts to a downstream pipeline, the ref value is usually a branch name, like main or development.

For merge request pipelines, the ref value is in the form of refs/merge-requests/<id>/head, where id is the merge request ID. You can retrieve this ref with the CI_MERGE_REQUEST_REF_PATH CI/CD variable. Do not use a branch name as the ref with merge request pipelines, because the downstream pipeline attempts to fetch artifacts from the latest branch pipeline.

To fetch the artifacts from the upstream merge request pipeline instead of the branch pipeline, pass this variable to the downstream pipeline using variable inheritance:

In a job in the upstream pipeline, save the artifacts using the artifacts keyword.

In the job that triggers the downstream pipeline, pass the $CI_MERGE_REQUEST_REF_PATH variable by using variable inheritance:

build_artifacts:
  stage: build
  script:
    - echo "This is a test artifact!" >> artifact.txt
  artifacts:
    paths:
      - artifact.txt

upstream_job:
  variables:
    UPSTREAM_REF: $CI_MERGE_REQUEST_REF_PATH
  trigger:
    project: my/downstream_project
    branch: my-branch

In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline by using needs:project. Set the ref to the UPSTREAM_REF variable, and job to the job in the upstream pipeline to fetch artifacts from:

test:
  stage: test
  script:
    - cat artifact.txt
  needs:
    - project: my/upstream_project
      job: build_artifacts
      ref: UPSTREAM_REF
      artifacts: true

This method works for fetching artifacts from a regular merge request parent pipeline, but fetching artifacts from merge results pipelines is not supported.

Use `rules` or `only`/`except` with multi-project pipelines

You can use CI/CD variables or the rules keyword to control job behavior for multi-project pipelines. When a downstream pipeline is triggered with the trigger keyword, the value of the $CI_PIPELINE_SOURCE predefined variable is pipeline for all its jobs.

If you use only/except to control job behavior, use the pipelines keyword.

Create multi-project pipelines by using the API

Moved to GitLab Free in 12.4.

When you use the CI_JOB_TOKEN to trigger pipelines, GitLab recognizes the source of the job token. The pipelines become related, so you can visualize their relationships on pipeline graphs.

These relationships are displayed in the pipeline graph by showing inbound and outbound connections for upstream and downstream pipeline dependencies.

When using:

CI/CD variables or rules to control job behavior, the value of the $CI_PIPELINE_SOURCE predefined variable is pipeline for multi-project pipeline triggered through the API with CI_JOB_TOKEN.
only/except to control job behavior, use the pipelines keyword.

Multi-project pipeline visualization (PREMIUM)

When your pipeline triggers a downstream pipeline, the downstream pipeline displays to the right of the pipeline graph.

In pipeline mini graphs, the downstream pipeline displays to the right of the mini graph.

10 KiB Raw Blame History