kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Add latest changes from gitlab-org/gitlab@master

Browse source
This commit is contained in:
GitLab Bot 2021-04-01 03:08:55 +00:00
parent 63eb68d335
commit 4223ed2e83
13 changed files with 123 additions and 122 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.gitlab/ci/rules.gitlab-ci.yml
View file

@ -101,6 +101,7 @@
- ".gitlab/ci/frontend.gitlab-ci.yml"
- ".gitlab/ci/build-images.gitlab-ci.yml"
- ".gitlab/ci/review.gitlab-ci.yml"
- "scripts/review_apps/base-config.yaml"
- "scripts/trigger-build"
.ci-qa-patterns: &ci-qa-patterns

2
app/controllers/concerns/milestone_actions.rb
View file

@ -20,7 +20,7 @@ module MilestoneActions
format.html { redirect_to milestone_redirect_path }
format.json do
render json: tabs_json("shared/milestones/_merge_requests_tab", {
merge_requests: @milestone.sorted_merge_requests(current_user), # rubocop:disable Gitlab/ModuleWithInstanceVariables
merge_requests: @milestone.sorted_merge_requests(current_user).preload_milestoneish_associations, # rubocop:disable Gitlab/ModuleWithInstanceVariables
show_project_name: Gitlab::Utils.to_boolean(params[:show_project_name])
})
end

2
app/models/merge_request.rb
View file

@ -359,6 +359,8 @@ class MergeRequest < ApplicationRecord
scope :preload_metrics, -> (relation) { preload(metrics: relation) }
scope :preload_project_and_latest_diff, -> { preload(:source_project, :latest_merge_request_diff) }
scope :preload_latest_diff_commit, -> { preload(latest_merge_request_diff: :merge_request_diff_commits) }
scope :preload_milestoneish_associations, -> { preload_routables.preload(:assignees, :labels) }
scope :with_web_entity_associations, -> { preload(:author, target_project: [:project_feature, group: [:route, :parent], namespace: :route]) }
scope :with_auto_merge_enabled, -> do

5
changelogs/unreleased/id-preload-associations-to-milestones-mrs.yml Normal file
View file

@ -0,0 +1,5 @@
---
title: 'Fix N + 1 for MilestonesController#merge_requests'
merge_request: 57980
author:
type: performance

20
doc/administration/troubleshooting/group_saml_scim.md
View file

@ -85,23 +85,3 @@ Adding a user:
SSO settings:
![OneLogin SSO settings](img/OneLogin-SSOsettings.png)
## ADFS
Setup SAML SSO URL:
![ADFS Setup SAML SSO URL](img/ADFS-saml-setup-sso-url.png)
Configure Assertions:
![ADFS Configure Assertions](img/ADFS-configure-assertions.png)
Configure NameID:
![ADFS ADFS-configure-NameID](img/ADFS-configure-NameID.png)
Determine Certificate Fingerprint:
| Via UI | Via Shell |
|--------|-----------|
| ![ADFS Determine Token Signing Certificate Fingerprint](img/ADFS-determine-token-signing-certificate-fingerprint.png) | ![ADFS Determine Token Signing Fingerprint From Shell](img/ADFS-determine-token-signing-fingerprint-from-shell.png) |

BIN
doc/administration/troubleshooting/img/ADFS-configure-NameID.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 22 KiB

BIN
doc/administration/troubleshooting/img/ADFS-configure-assertions.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 10 KiB

BIN
doc/administration/troubleshooting/img/ADFS-determine-token-signing-certificate-fingerprint.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 36 KiB

BIN
doc/administration/troubleshooting/img/ADFS-determine-token-signing-fingerprint-from-shell.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 28 KiB

BIN
doc/administration/troubleshooting/img/ADFS-saml-setup-sso-url.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 30 KiB

4
doc/ci/docker/using_docker_build.md
View file

@ -623,7 +623,7 @@ fails, because the file is mounted as read-only. Do not change it from
read-only, because problems occur.
Here is an example of `/opt/.docker/config.json` that follows the
[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determining-your-docker_auth_config-data)
[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determine-your-docker_auth_config-data)
documentation:
```json
@ -683,7 +683,7 @@ to include the file.
### Option 3: Use `DOCKER_AUTH_CONFIG`
If you already have
[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determining-your-docker_auth_config-data)
[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determine-your-docker_auth_config-data)
defined, you can use the variable and save it in
`~/.docker/config.json`.

187
doc/ci/docker/using_docker_images.md
View file

@ -9,23 +9,22 @@ type: concepts, howto
You can run your CI/CD jobs in separate, isolated Docker containers.
When you run a Docker container on your local machine, it acts as a reproducible build environment.
You can run tests in the container, instead of testing on a dedicated CI/CD server.
If you run Docker on your local machine, you can run tests in the container,
rather than testing on a dedicated CI/CD server.
To run CI/CD jobs in a Docker container, you need to:
- Register a runner that uses the Docker executor. Then all jobs run in a Docker container.
- Specify an image in your `.gitlab-ci.yml` file. The runner creates a container from this image
and runs the jobs in it.
- Optional. Specify other images in your `.gitlab-ci.yml` file. These containers are known as
["services"](../services/index.md) and you can use them to run services like MySQL separately.
1. Register a runner so that all jobs run in Docker containers. Do this by choosing the Docker executor during registration.
1. Specify which container to run the jobs in. Do this by specifying an image in your `.gitlab-ci.yml` file.
1. Optional. Run other services, like MySQL, in containers. Do this by specifying [services](../services/index.md)
in your `.gitlab-ci.yml` file.
## Register a runner that uses the Docker executor
To use GitLab Runner with Docker you need to [register a runner](https://docs.gitlab.com/runner/register/)
that uses the Docker executor.
In this example, we first set up a temporary template to supply the services:
This example shows how to set up a temporary template to supply services:
```shell
cat > /tmp/test-config.template.toml << EOF
@ -57,20 +56,20 @@ accessible during the build process.
## What is an image
The `image` keyword is the name of the Docker image the Docker executor
runs to perform the CI tasks.
uses to run CI/CD jobs.
By default, the executor pulls images only from [Docker Hub](https://hub.docker.com/).
However, you can configure the location in the `gitlab-runner/config.toml` file. For example,
you can set the [Docker pull policy](https://docs.gitlab.com/runner/executors/docker.html#how-pull-policies-work)
By default, the executor pulls images from [Docker Hub](https://hub.docker.com/).
However, you can configure the registry location in the `gitlab-runner/config.toml` file.
For example, you can set the [Docker pull policy](https://docs.gitlab.com/runner/executors/docker.html#how-pull-policies-work)
to use local images.
For more information about images and Docker Hub, read
For more information about images and Docker Hub, see
the [Docker Fundamentals](https://docs.docker.com/engine/understanding-docker/) documentation.
## Define `image` in the `.gitlab-ci.yml` file
You can define an image that's used for all jobs, and a list of
services that you want to use during build time:
services that you want to use during runtime:
```yaml
default:
@ -97,58 +96,57 @@ The image name must be in one of the following formats:
> Introduced in GitLab and GitLab Runner 9.4.
When configuring the `image` or `services` entries, you can use a string or a map as
options:
You can use a string or a map for the `image` or `services` entries:
- When using a string as an option, it must be the full name of the image to use
(including the Registry part if you want to download the image from a Registry
- Strings must include the full image name
(including the registry, if you want to download the image from a registry
other than Docker Hub).
- When using a map as an option, then it must contain at least the `name`
option, which is the same name of the image as used for the string setting.
- Maps must contain at least the `name` option,
which is the same image name as used for the string setting.
For example, the following two definitions are equal:
1. Using a string as an option to `image` and `services`:
- A string for `image` and `services`:
```yaml
image: "registry.example.com/my/image:latest"
```yaml
image: "registry.example.com/my/image:latest"
services:
- postgresql:9.4
- redis:latest
```
services:
- postgresql:9.4
- redis:latest
```
1. Using a map as an option to `image` and `services`. The use of `image:name` is
required:
- A map for `image` and `services`. The `image:name` is
required:
```yaml
image:
name: "registry.example.com/my/image:latest"
```yaml
image:
name: "registry.example.com/my/image:latest"
services:
- name: postgresql:9.4
- name: redis:latest
```
services:
- name: postgresql:9.4
- name: redis:latest
```
### Available settings for `image`
> Introduced in GitLab and GitLab Runner 9.4.
| Setting | Required | GitLab version | Description |
|------------|----------|----------------| ----------- |
| `name` | yes, when used with any other option | 9.4 |Full name of the image to use. It should contain the Registry part if needed. |
| `entrypoint` | no | 9.4 |Command or script to execute as the container's entrypoint. It's translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, where each shell token is a separate string in the array. |
| Setting | Required | Description |
|------------|----------| ----------- |
| `name` | Yes, when used with any other option. | Full name of the image. It should contain the registry part if needed. |
| `entrypoint` | No. | Command or script to execute as the container's entrypoint. It's translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, where each shell token is a separate string in the array. |
### Overriding the entrypoint of an image
> Introduced in GitLab and GitLab Runner 9.4. Read more about the [extended configuration options](../docker/using_docker_images.md#extended-docker-configuration-options).
Before showing the available entrypoint override methods, let's describe
Before explaining the available entrypoint override methods, let's describe
how the runner starts. It uses a Docker image for the containers used in the
CI/CD jobs:
1. The runner starts a Docker container using the defined entrypoint (default
from `Dockerfile` that may be overridden in `.gitlab-ci.yml`)
1. The runner starts a Docker container using the defined entrypoint. The default
from `Dockerfile` that may be overridden in the `.gitlab-ci.yml` file.
1. The runner attaches itself to a running container.
1. The runner prepares a script (the combination of
[`before_script`](../yaml/README.md#before_script),
@ -157,14 +155,13 @@ CI/CD jobs:
1. The runner sends the script to the container's shell `stdin` and receives the
output.
To override the entrypoint of a Docker image, you should
define an empty `entrypoint` in `.gitlab-ci.yml`, so the runner does not start
a useless shell layer. However, that does not work for all Docker versions, and
you should check which one your runner is using:
To override the entrypoint of a Docker image,
define an empty `entrypoint` in the `.gitlab-ci.yml` file, so the runner does not start
a useless shell layer. However, that does not work for all Docker versions.
- _If Docker 17.06 or later is used,_ the `entrypoint` can be set to an empty value.
- _If Docker 17.03 or previous versions are used,_ the `entrypoint` can be set to
`/bin/sh -c`, `/bin/bash -c` or an equivalent shell available in the image.
- For Docker 17.06 and later, the `entrypoint` can be set to an empty value.
- For Docker 17.03 and earlier, the `entrypoint` can be set to
`/bin/sh -c`, `/bin/bash -c`, or an equivalent shell available in the image.
The syntax of `image:entrypoint` is similar to [Dockerfile's `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint).
@ -184,7 +181,7 @@ With the extended Docker configuration options, instead of:
You can now define an `entrypoint` in the `.gitlab-ci.yml` file.
**For Docker 17.06+:**
**For Docker 17.06 and later:**
```yaml
image:
@ -192,7 +189,7 @@ image:
entrypoint: [""]
```
**For Docker =< 17.03:**
**For Docker 17.03 and earlier:**
```yaml
image:
@ -217,40 +214,38 @@ that runner.
To access private container registries, the GitLab Runner process can use:
- [Statically defined credentials](#using-statically-defined-credentials). That is, a username and password for a specific registry.
- [Credentials Store](#use-a-credentials-store). For more information, read [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credentials-store).
- [Credential Helpers](#use-credential-helpers). For more information, read [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credential-helpers).
- [Statically defined credentials](#use-statically-defined-credentials). That is, a username and password for a specific registry.
- [Credentials Store](#use-a-credentials-store). For more information, see [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credentials-store).
- [Credential Helpers](#use-credential-helpers). For more information, see [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credential-helpers).
To define which should be used, the GitLab Runner process reads the configuration in the following order:
To define which option should be used, the runner process reads the configuration in this order:
- `DOCKER_AUTH_CONFIG` variable provided as either:
- A [CI/CD variable](../variables/README.md) in `.gitlab-ci.yml`.
- A project's variables stored on the projects **Settings > CI/CD** page.
- `DOCKER_AUTH_CONFIG` variable provided as environment variable in `config.toml` of the runner.
- `config.json` file placed in `$HOME/.docker` directory of the user running GitLab Runner process.
If the `--user` flag is provided to run the GitLab Runner child processes as unprivileged user,
the home directory of the main GitLab Runner process user is used.
- A `DOCKER_AUTH_CONFIG` variable provided as either:
- A [CI/CD variable](../variables/README.md) in the `.gitlab-ci.yml` file.
- A project's variables stored on the project's **Settings > CI/CD** page.
- A `DOCKER_AUTH_CONFIG` variable provided as environment variable in the runner's `config.toml` file.
- A `config.json` file in `$HOME/.docker` directory of the user running the process.
If the `--user` flag is provided to run the child processes as unprivileged user,
the home directory of the main runner process user is used.
GitLab Runner reads this configuration **only** from `config.toml` and ignores it if
it's provided as an environment variable. This is because GitLab Runner uses **only**
`config.toml` configuration and does not interpolate **any** environment variables at
The runner reads this configuration **only** from the `config.toml` file and ignores it if
it's provided as a CI/CD variable. This is because the runner uses **only**
`config.toml` configuration and does not interpolate **any** CI/CD variables at
runtime.
### Requirements and limitations
- This feature requires GitLab Runner **1.8** or higher.
- For GitLab Runner versions **>= 0.6, <1.8** there was a partial
support for using private registries, which required manual configuration
of credentials on runner's host. We recommend to upgrade your runner to
at least version **1.8** if you want to use private registries.
- Available for [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html)
in GitLab Runner 13.1 and later.
- [Credentials Store](#use-a-credentials-store) and [Credential Helpers](#use-credential-helpers) require binaries to be added to the GitLab Runner's `$PATH`, and require access to do so. Therefore, these features are not available on shared runners, or any other runner where the user does not have access to the environment where the runner is installed.
- [Credentials Store](#use-a-credentials-store) and [Credential Helpers](#use-credential-helpers)
require binaries to be added to the GitLab Runner `$PATH`, and require access to do so. Therefore,
these features are not available on shared runners, or any other runner where the user does not
have access to the environment where the runner is installed.
### Using statically-defined credentials
### Use statically-defined credentials
There are two approaches that you can take in order to access a
private registry. Both require setting the environment variable
There are two approaches that you can take to access a
private registry. Both require setting the CI/CD variable
`DOCKER_AUTH_CONFIG` with appropriate authentication information.
1. Per-job: To configure one job to access a private registry, add
@ -261,7 +256,7 @@ private registry. Both require setting the environment variable
See below for examples of each.
#### Determining your `DOCKER_AUTH_CONFIG` data
#### Determine your `DOCKER_AUTH_CONFIG` data
As an example, let's assume you want to use the `registry.example.com:5000/private/image:latest`
image. This image is private and requires you to sign in to a private container
@ -292,7 +287,7 @@ Use one of the following methods to determine the value of `DOCKER_AUTH_CONFIG`:
docker logout registry.example.com:5000
```
- In some setups, it's possible that Docker client uses the available system key
- In some setups, it's possible the Docker client uses the available system key
store to store the result of `docker login`. In that case, it's impossible to
read `~/.docker/config.json`, so you must prepare the required
base64-encoded version of `${username}:${password}` and create the Docker
@ -318,7 +313,7 @@ Use one of the following methods to determine the value of `DOCKER_AUTH_CONFIG`:
}
```
#### Configuring a job
#### Configure a job
To configure a single job with access for `registry.example.com:5000`,
follow these steps:
@ -337,7 +332,7 @@ follow these steps:
```
1. You can now use any private image from `registry.example.com:5000` defined in
`image` and/or `services` in your `.gitlab-ci.yml` file:
`image` or `services` in your `.gitlab-ci.yml` file:
```yaml
image: registry.example.com:5000/namespace/image:tag
@ -351,19 +346,19 @@ registries to the `"auths"` hash as described above.
The full `hostname:port` combination is required everywhere
for the runner to match the `DOCKER_AUTH_CONFIG`. For example, if
`registry.example.com:5000/namespace/image:tag` is specified in `.gitlab-ci.yml`,
`registry.example.com:5000/namespace/image:tag` is specified in the `.gitlab-ci.yml` file,
then the `DOCKER_AUTH_CONFIG` must also specify `registry.example.com:5000`.
Specifying only `registry.example.com` does not work.
### Configuring a runner
If you have many pipelines that access the same registry, it is
probably better to set up registry access at the runner level. This
If you have many pipelines that access the same registry, you should
set up registry access at the runner level. This
allows pipeline authors to have access to a private registry just by
running a job on the appropriate runner. It also makes registry
changes and credential rotations much simpler.
running a job on the appropriate runner. It also helps simplify registry
changes and credential rotations.
Of course this means that any job on that runner can access the
This means that any job on that runner can access the
registry with the same privilege, even across projects. If you need to
control access to the registry, you need to be sure to control
access to the runner.
@ -388,10 +383,10 @@ To add `DOCKER_AUTH_CONFIG` to a runner:
### Use a Credentials Store
To configure a credentials store:
To configure a Credentials Store:
1. To use a credentials store, you need an external helper program to interact with a specific keychain or external store.
Make sure the helper program is available in GitLab Runner `$PATH`.
1. To use a Credentials Store, you need an external helper program to interact with a specific keychain or external store.
Make sure the helper program is available in the GitLab Runner `$PATH`.
1. Make GitLab Runner use it. There are two ways to accomplish this. Either:
@ -416,14 +411,14 @@ pulling from Docker Hub fails. Docker daemon tries to use the same credentials f
### Use Credential Helpers
> Support for using Credential Helpers was added in GitLab Runner 12.0
> Introduced in GitLab Runner 12.0.
As an example, let's assume that you want to use the `aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest`
image. This image is private and requires you to log in into a private container registry.
To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow these steps:
1. Make sure `docker-credential-ecr-login` is available in GitLab Runner's `$PATH`.
1. Make sure `docker-credential-ecr-login` is available in the GitLab Runner `$PATH`.
1. Have any of the following [AWS credentials setup](https://github.com/awslabs/amazon-ecr-credential-helper#aws-credentials).
Make sure that GitLab Runner can access the credentials.
1. Make GitLab Runner use it. There are two ways to accomplish this. Either:
@ -440,9 +435,9 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th
}
```
This configures Docker to use the credential helper for a specific registry.
This configures Docker to use the Credential Helper for a specific registry.
or
Instead, you can configure Docker to use the Credential Helper for all Amazon Elastic Container Registry (ECR) registries:
```json
{
@ -450,10 +445,8 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th
}
```
This configures Docker to use the credential helper for all Amazon Elastic Container Registry (ECR) registries.
- Or, if you're running self-managed runners,
add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`.
add the previous JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`.
GitLab Runner reads this configuration file and uses the needed helper for this
specific repository.
@ -464,8 +457,8 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th
image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest
```
In the example above, GitLab Runner looks at `aws_account_id.dkr.ecr.region.amazonaws.com` for the
In the example, GitLab Runner looks at `aws_account_id.dkr.ecr.region.amazonaws.com` for the
image `private/image:latest`.
You can add configuration for as many registries as you want, adding more
registries to the `"credHelpers"` hash as described above.
registries to the `"credHelpers"` hash.

24
spec/requests/groups/milestones_controller_spec.rb
View file

@ -4,8 +4,8 @@ require 'spec_helper'
RSpec.describe Groups::MilestonesController do
context 'N+1 DB queries' do
let(:user) { create(:user) }
let!(:public_group) { create(:group, :public) }
let_it_be(:user) { create(:user) }
let_it_be(:public_group) { create(:group, :public) }
let!(:public_project_with_private_issues_and_mrs) do
create(:project, :public, :issues_private, :merge_requests_private, group: public_group)
@ -53,5 +53,25 @@ RSpec.describe Groups::MilestonesController do
expect { get show_path }.not_to exceed_all_query_limit(control)
end
end
describe 'GET #merge_requests' do
let(:milestone) { create(:milestone, group: public_group) }
let(:project) { create(:project, :public, :merge_requests_enabled, :issues_enabled, group: public_group) }
let!(:merge_request) { create(:merge_request, milestone: milestone, source_project: project) }
def perform_request
get merge_requests_group_milestone_path(public_group, milestone, format: :json)
end
it 'avoids N+1 database queries' do
perform_request # warm up the cache
control_count = ActiveRecord::QueryRecorder.new { perform_request }.count
create(:merge_request, milestone: milestone, source_project: project, source_branch: 'fix')
expect { perform_request }.not_to exceed_query_limit(control_count)
end
end
end
end