kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Add latest changes from gitlab-org/gitlab@master

Browse source
This commit is contained in:
GitLab Bot 2019-10-01 06:06:13 +00:00
parent ec60358d52
commit 1dd77c7113
4 changed files with 78 additions and 68 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

109
doc/ci/docker/using_docker_build.md
View file

@ -117,10 +117,10 @@ not without its own challenges:
history. Concurrent jobs work fine because every build gets it's own
instance of Docker engine so they won't conflict with each other. But this
also means jobs can be slower because there's no caching of layers.
- By default, `docker:dind` uses `--storage-driver vfs` which is the slowest
form offered. To use a different driver, see
[Using the overlayfs driver](#using-the-overlayfs-driver).
- Since the `docker:dind` container and the runner container don't share their
- By default, Docker 17.09 and higher uses `--storage-driver overlay2` which is
the recommended storage driver. See [Using the overlayfs driver](#using-the-overlayfs-driver)
for details.
- Since the `docker:19.03.1-dind` container and the Runner container don't share their
root filesystem, the job's working directory can be used as a mount point for
child containers. For example, if you have files you want to share with a
child container, you may create a subdirectory under `/builds/$CI_PROJECT_PATH`
@ -156,7 +156,6 @@ docker-in-docker service and
support this.
1. Install [GitLab Runner](https://docs.gitlab.com/runner/install).
1. Register GitLab Runner from the command line to use `docker` and `privileged`
mode:
@ -218,13 +217,10 @@ support this.
# https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services.
#
# Note that if you're using the Kubernetes executor, the variable
# should be set to tcp://localhost:2376/ because of how the
# should be set to tcp://localhost:2376 because of how the
# Kubernetes executor connects services to the job container
# DOCKER_HOST: tcp://localhost:2376/
# DOCKER_HOST: tcp://localhost:2376
#
# When using dind, it's wise to use the overlayfs driver for
# improved performance.
DOCKER_DRIVER: overlay2
# Specify to Docker where to create the certificates, Docker will
# create them automatically on boot, and will create
# `/certs/client` that will be shared between the service and job
@ -283,15 +279,12 @@ variables:
# https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services
#
# Note that if you're using the Kubernetes executor, the variable should be set to
# tcp://localhost:2375/ because of how the Kubernetes executor connects services
# tcp://localhost:2375 because of how the Kubernetes executor connects services
# to the job container
# DOCKER_HOST: tcp://localhost:2375/
# DOCKER_HOST: tcp://localhost:2375
#
# For non-Kubernetes executors, we use tcp://docker:2375/
DOCKER_HOST: tcp://docker:2375/
# When using dind, it's wise to use the overlayfs driver for
# improved performance.
DOCKER_DRIVER: overlay2
# For non-Kubernetes executors, we use tcp://docker:2375
DOCKER_HOST: tcp://docker:2375
#
# This will instruct Docker not to start over TLS.
DOCKER_TLS_CERTDIR: ""
@ -317,13 +310,12 @@ container so that Docker is available in the context of that image.
NOTE: **Note:**
If you bind the Docker socket [when using GitLab Runner 11.11 or
newer](https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1261),
you can no longer use `docker:dind` as a service because volume bindings
you can no longer use `docker:19.03.1-dind` as a service because volume bindings
are done to the services as well, making these incompatible.
In order to do that, follow the steps:
1. Install [GitLab Runner](https://docs.gitlab.com/runner/install).
1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`:
```bash
@ -332,14 +324,14 @@ In order to do that, follow the steps:
--registration-token REGISTRATION_TOKEN \
--executor docker \
--description "My Docker Runner" \
--docker-image "docker:stable" \
--docker-image "docker:19.03.1" \
--docker-volumes /var/run/docker.sock:/var/run/docker.sock
```
The above command will register a new Runner to use the special
`docker:stable` image which is provided by Docker. **Notice that it's using
the Docker daemon of the Runner itself, and any containers spawned by docker
commands will be siblings of the Runner rather than children of the runner.**
`docker:19.03.1` image which is provided by Docker. **Notice that it's using
the Docker daemon of the Runner itself, and any containers spawned by Docker
commands will be siblings of the Runner rather than children of the Runner.**
This may have complications and limitations that are unsuitable for your workflow.
The above command will create a `config.toml` entry similar to this:
@ -351,7 +343,7 @@ In order to do that, follow the steps:
executor = "docker"
[runners.docker]
tls_verify = false
image = "docker:stable"
image = "docker:19.03.1"
privileged = false
disable_cache = false
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
@ -360,10 +352,11 @@ In order to do that, follow the steps:
```
1. You can now use `docker` in the build script (note that you don't need to
include the `docker:dind` service as when using the Docker in Docker executor):
include the `docker:19.03.1-dind` service as when using the Docker in Docker
executor):
```yaml
image: docker:stable
image: docker:19.03.1
before_script:
- docker info
@ -417,14 +410,15 @@ any image that's used with the `--cache-from` argument must first be pulled
Here's a simple `.gitlab-ci.yml` file showing how Docker caching can be utilized:
```yaml
image: docker:stable
image: docker:19.03.1
services:
- docker:dind
- docker:19.03.1-dind
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
# Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled
DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_CERTDIR: "/certs"
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
@ -597,7 +591,6 @@ assuming you have it configured with [TLS enabled](#tls-enabled):
# `/certs/client` that will be shared between the service and
# build container.
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_DRIVER: overlay2
stage: build
script:
- docker build -t my-docker-image .
@ -618,37 +611,36 @@ If you're using docker-in-docker on your Runners, this is how your `.gitlab-ci.y
could look like:
```yaml
build:
image: docker:stable
services:
- docker:dind
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
stage: build
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build -t $CI_REGISTRY/group/project/image:latest .
- docker push $CI_REGISTRY/group/project/image:latest
build:
image: docker:19.03.1
stage: build
services:
- docker:19.03.1-dind
variables:
# Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled
DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_CERTDIR: "/certs"
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build -t $CI_REGISTRY/group/project/image:latest .
- docker push $CI_REGISTRY/group/project/image:latest
```
You can also make use of [other variables](../variables/README.md) to avoid hardcoding:
```yaml
services:
- docker:dind
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
build:
image: docker:19.03.1
stage: build
services:
- docker:19.03.1-dind
variables:
# Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled
DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_CERTDIR: "/certs"
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build -t $IMAGE_TAG .
- docker push $IMAGE_TAG
```
@ -667,9 +659,9 @@ when needed. Changes to `master` also get tagged as `latest` and deployed using
an application-specific deploy script:
```yaml
image: docker:stable
image: docker:19.03.1
services:
- docker:dind
- docker:19.03.1-dind
stages:
- build
@ -678,8 +670,9 @@ stages:
- deploy
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
# Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled
DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_CERTDIR: "/certs"
CONTAINER_TEST_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
CONTAINER_RELEASE_IMAGE: $CI_REGISTRY_IMAGE:latest

17
lib/gitlab/utils/inline_hash.rb
View file

@ -11,13 +11,15 @@ module Gitlab
#
# {
# 'root_param' => 'Root',
# 12 => 'number',
# symbol: 'symbol',
# nested_param: {
# key: 'Value'
# },
# 'very' => {
# 'deep' => {
# 'nested' => {
# 'param' => 'Deep nested value'
# 12 => 'Deep nested value'
# }
# }
# }
@ -28,14 +30,21 @@ module Gitlab
#
# {
# 'root_param' => 'Root',
# 12 => 'number',
# symbol: symbol,
# 'nested_param.key' => 'Value',
# 'very.deep.nested.param' => 'Deep nested value'
# 'very.deep.nested.12' => 'Deep nested value'
# }
#
def merge_keys(hash, prefix: nil, connector: '.')
result = {}
base_prefix = prefix ? "#{prefix}#{connector}" : ''
pairs = hash.map { |key, value| ["#{base_prefix}#{key}", value] }
pairs =
if prefix
base_prefix = "#{prefix}#{connector}"
hash.map { |key, value| ["#{base_prefix}#{key}", value] }
else
hash.to_a
end
until pairs.empty?
key, value = pairs.shift

18
spec/lib/gitlab/utils/inline_hash_spec.rb
View file

@ -9,9 +9,11 @@ describe Gitlab::Utils::InlineHash do
let(:source) do
{
nested_param: {
key: 'Value'
key: :Value
},
'root_param' => 'Root',
unnested_symbol_key: :unnested_symbol_value,
12 => 22,
'very' => {
'deep' => {
'nested' => {
@ -24,15 +26,17 @@ describe Gitlab::Utils::InlineHash do
it 'transforms a nested hash into a one-level hash' do
is_expected.to eq(
'nested_param.key' => 'Value',
'nested_param.key' => :Value,
'root_param' => 'Root',
:unnested_symbol_key => :unnested_symbol_value,
12 => 22,
'very.deep.nested.param' => 'Deep nested value'
)
end
it 'retains key insertion order' do
expect(subject.keys)
.to eq(%w(nested_param.key root_param very.deep.nested.param))
.to eq(['nested_param.key', 'root_param', :unnested_symbol_key, 12, 'very.deep.nested.param'])
end
context 'with a custom connector' do
@ -40,8 +44,10 @@ describe Gitlab::Utils::InlineHash do
it 'uses the connector to merge keys' do
is_expected.to eq(
'nested_param::key' => 'Value',
'nested_param::key' => :Value,
'root_param' => 'Root',
:unnested_symbol_key => :unnested_symbol_value,
12 => 22,
'very::deep::nested::param' => 'Deep nested value'
)
end
@ -52,8 +58,10 @@ describe Gitlab::Utils::InlineHash do
it 'prefixes all the keys' do
is_expected.to eq(
'options.nested_param.key' => 'Value',
'options.nested_param.key' => :Value,
'options.root_param' => 'Root',
'options.unnested_symbol_key' => :unnested_symbol_value,
'options.12' => 22,
'options.very.deep.nested.param' => 'Deep nested value'
)
end

2
spec/lib/gitlab/utils/safe_inline_hash_spec.rb
View file

@ -3,7 +3,7 @@
require 'fast_spec_helper'
describe Gitlab::Utils::SafeInlineHash do
context '.merge_keys!' do
describe '.merge_keys!' do
let(:source) { { 'foo' => { 'bar' => 'baz' } } }
let(:validator) { instance_double(Gitlab::Utils::DeepSize, valid?: valid) }