kotovalexarian-likes-gitlab
/
gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Fix whitespace in ci docs 

Many code blocks are 4spaced, and they render in GitLab
without coloring as a result, even though they are
fenced with a language label. If in a list, other items
will render as being in a code block too, even if not
meant to. This fixes all these issues for most docs in
/ci, and cleans up other minor whitespace issues too.
This commit is contained in:
Marcel Amirault 2019-08-01 02:29:16 +00:00 committed by Evan Read
parent cfb7f11644
commit a696695f31
7 changed files with 449 additions and 445 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

178
doc/ci/ci_cd_for_external_repos/bitbucket_integration.md
View File

@ -14,9 +14,9 @@ To use GitLab CI/CD with a Bitbucket Cloud repository:
1. In GitLab create a **CI/CD for external repo**, select **Repo by URL** and 1. In GitLab create a **CI/CD for external repo**, select **Repo by URL** and
create the project. create the project.
![Create project](img/external_repository.png) ![Create project](img/external_repository.png)
GitLab will import the repository and enable [Pull Mirroring][pull-mirroring]. GitLab will import the repository and enable [Pull Mirroring][pull-mirroring].
1. In GitLab create a 1. In GitLab create a
[Personal Access Token](../../user/profile/personal_access_tokens.md) [Personal Access Token](../../user/profile/personal_access_tokens.md)
@ -26,19 +26,19 @@ To use GitLab CI/CD with a Bitbucket Cloud repository:
1. In Bitbucket, from **Settings > Webhooks**, create a new web hook to notify 1. In Bitbucket, from **Settings > Webhooks**, create a new web hook to notify
GitLab of new commits. GitLab of new commits.
The web hook URL should be set to the GitLab API to trigger pull mirroring, The web hook URL should be set to the GitLab API to trigger pull mirroring,
using the Personal Access Token we just generated for authentication. using the Personal Access Token we just generated for authentication.
```text ```text
https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN>
``` ```
The web hook Trigger should be set to 'Repository Push'. The web hook Trigger should be set to 'Repository Push'.
![Bitbucket Cloud webhook](img/bitbucket_webhook.png) ![Bitbucket Cloud webhook](img/bitbucket_webhook.png)
After saving, test the web hook by pushing a change to your Bitbucket After saving, test the web hook by pushing a change to your Bitbucket
repository. repository.
1. In Bitbucket, create an **App Password** from **Bitbucket Settings > App 1. In Bitbucket, create an **App Password** from **Bitbucket Settings > App
Passwords** to authenticate the build status script setting commit build Passwords** to authenticate the build status script setting commit build
@ -49,104 +49,104 @@ To use GitLab CI/CD with a Bitbucket Cloud repository:
1. In GitLab, from **Settings > CI/CD > Environment variables**, add variables to allow 1. In GitLab, from **Settings > CI/CD > Environment variables**, add variables to allow
communication with Bitbucket via the Bitbucket API: communication with Bitbucket via the Bitbucket API:
`BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above. `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above.
`BITBUCKET_USERNAME`: the username of the Bitbucket account. `BITBUCKET_USERNAME`: the username of the Bitbucket account.
`BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ. `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ.
`BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ. `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ.
1. In Bitbucket, add a script to push the pipeline status to Bitbucket. 1. In Bitbucket, add a script to push the pipeline status to Bitbucket.
> Note: changes made in GitLab will be overwritten by any changes made > Note: changes made in GitLab will be overwritten by any changes made
> upstream in Bitbucket. > upstream in Bitbucket.
Create a file `build_status` and insert the script below and run Create a file `build_status` and insert the script below and run
`chmod +x build_status` in your terminal to make the script executable. `chmod +x build_status` in your terminal to make the script executable.
```bash ```bash
#!/usr/bin/env bash #!/usr/bin/env bash
# Push GitLab CI/CD build status to Bitbucket Cloud # Push GitLab CI/CD build status to Bitbucket Cloud
if [ -z "$BITBUCKET_ACCESS_TOKEN" ]; then if [ -z "$BITBUCKET_ACCESS_TOKEN" ]; then
echo "ERROR: BITBUCKET_ACCESS_TOKEN is not set" echo "ERROR: BITBUCKET_ACCESS_TOKEN is not set"
exit 1 exit 1
fi fi
if [ -z "$BITBUCKET_USERNAME" ]; then if [ -z "$BITBUCKET_USERNAME" ]; then
echo "ERROR: BITBUCKET_USERNAME is not set" echo "ERROR: BITBUCKET_USERNAME is not set"
exit 1 exit 1
fi fi
if [ -z "$BITBUCKET_NAMESPACE" ]; then if [ -z "$BITBUCKET_NAMESPACE" ]; then
echo "Setting BITBUCKET_NAMESPACE to $CI_PROJECT_NAMESPACE" echo "Setting BITBUCKET_NAMESPACE to $CI_PROJECT_NAMESPACE"
BITBUCKET_NAMESPACE=$CI_PROJECT_NAMESPACE BITBUCKET_NAMESPACE=$CI_PROJECT_NAMESPACE
fi fi
if [ -z "$BITBUCKET_REPOSITORY" ]; then if [ -z "$BITBUCKET_REPOSITORY" ]; then
echo "Setting BITBUCKET_REPOSITORY to $CI_PROJECT_NAME" echo "Setting BITBUCKET_REPOSITORY to $CI_PROJECT_NAME"
BITBUCKET_REPOSITORY=$CI_PROJECT_NAME BITBUCKET_REPOSITORY=$CI_PROJECT_NAME
fi fi
BITBUCKET_API_ROOT="https://api.bitbucket.org/2.0" BITBUCKET_API_ROOT="https://api.bitbucket.org/2.0"
BITBUCKET_STATUS_API="$BITBUCKET_API_ROOT/repositories/$BITBUCKET_NAMESPACE/$BITBUCKET_REPOSITORY/commit/$CI_COMMIT_SHA/statuses/build" BITBUCKET_STATUS_API="$BITBUCKET_API_ROOT/repositories/$BITBUCKET_NAMESPACE/$BITBUCKET_REPOSITORY/commit/$CI_COMMIT_SHA/statuses/build"
BITBUCKET_KEY="ci/gitlab-ci/$CI_JOB_NAME" BITBUCKET_KEY="ci/gitlab-ci/$CI_JOB_NAME"
case "$BUILD_STATUS" in case "$BUILD_STATUS" in
running) running)
BITBUCKET_STATE="INPROGRESS" BITBUCKET_STATE="INPROGRESS"
BITBUCKET_DESCRIPTION="The build is running!" BITBUCKET_DESCRIPTION="The build is running!"
;; ;;
passed) passed)
BITBUCKET_STATE="SUCCESSFUL" BITBUCKET_STATE="SUCCESSFUL"
BITBUCKET_DESCRIPTION="The build passed!" BITBUCKET_DESCRIPTION="The build passed!"
;; ;;
failed) failed)
BITBUCKET_STATE="FAILED" BITBUCKET_STATE="FAILED"
BITBUCKET_DESCRIPTION="The build failed." BITBUCKET_DESCRIPTION="The build failed."
;; ;;
esac esac
echo "Pushing status to $BITBUCKET_STATUS_API..." echo "Pushing status to $BITBUCKET_STATUS_API..."
curl --request POST $BITBUCKET_STATUS_API \ curl --request POST $BITBUCKET_STATUS_API \
--user $BITBUCKET_USERNAME:$BITBUCKET_ACCESS_TOKEN \ --user $BITBUCKET_USERNAME:$BITBUCKET_ACCESS_TOKEN \
--header "Content-Type:application/json" \ --header "Content-Type:application/json" \
--silent \ --silent \
--data "{ \"state\": \"$BITBUCKET_STATE\", \"key\": \"$BITBUCKET_KEY\", \"description\": --data "{ \"state\": \"$BITBUCKET_STATE\", \"key\": \"$BITBUCKET_KEY\", \"description\":
\"$BITBUCKET_DESCRIPTION\",\"url\": \"$CI_PROJECT_URL/-/jobs/$CI_JOB_ID\" }" \"$BITBUCKET_DESCRIPTION\",\"url\": \"$CI_PROJECT_URL/-/jobs/$CI_JOB_ID\" }"
``` ```
1. Still in Bitbucket, create a `.gitlab-ci.yml` file to use the script to push 1. Still in Bitbucket, create a `.gitlab-ci.yml` file to use the script to push
pipeline success and failures to Bitbucket. pipeline success and failures to Bitbucket.
```yaml ```yaml
stages: stages:
- test - test
- ci_status - ci_status
unit-tests: unit-tests:
script: script:
- echo "Success. Add your tests!" - echo "Success. Add your tests!"
success: success:
stage: ci_status stage: ci_status
before_script: before_script:
- "" - ""
after_script: after_script:
- "" - ""
script: script:
- BUILD_STATUS=passed BUILD_KEY=push ./build_status - BUILD_STATUS=passed BUILD_KEY=push ./build_status
when: on_success when: on_success
failure: failure:
stage: ci_status stage: ci_status
before_script: before_script:
- "" - ""
after_script: after_script:
- "" - ""
script: script:
- BUILD_STATUS=failed BUILD_KEY=push ./build_status - BUILD_STATUS=failed BUILD_KEY=push ./build_status
when: on_failure when: on_failure
``` ```
GitLab is now configured to mirror changes from Bitbucket, run CI/CD pipelines GitLab is now configured to mirror changes from Bitbucket, run CI/CD pipelines
configured in `.gitlab-ci.yml` and push the status to Bitbucket. configured in `.gitlab-ci.yml` and push the status to Bitbucket.

320
doc/ci/docker/using_docker_build.md
View File

@ -48,42 +48,42 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user.
1. During GitLab Runner installation select `shell` as method of executing job scripts or use command: 1. During GitLab Runner installation select `shell` as method of executing job scripts or use command:
```bash ```bash
sudo gitlab-runner register -n \ sudo gitlab-runner register -n \
--url https://gitlab.com/ \ --url https://gitlab.com/ \
--registration-token REGISTRATION_TOKEN \ --registration-token REGISTRATION_TOKEN \
--executor shell \ --executor shell \
--description "My Runner" --description "My Runner"
``` ```
1. Install Docker Engine on server. 1. Install Docker Engine on server.
For more information how to install Docker Engine on different systems For more information how to install Docker Engine on different systems
checkout the [Supported installations](https://docs.docker.com/engine/installation/). checkout the [Supported installations](https://docs.docker.com/engine/installation/).
1. Add `gitlab-runner` user to `docker` group: 1. Add `gitlab-runner` user to `docker` group:
```bash ```bash
sudo usermod -aG docker gitlab-runner sudo usermod -aG docker gitlab-runner
``` ```
1. Verify that `gitlab-runner` has access to Docker: 1. Verify that `gitlab-runner` has access to Docker:
```bash ```bash
sudo -u gitlab-runner -H docker info sudo -u gitlab-runner -H docker info
``` ```
You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`: You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`:
```yaml ```yaml
before_script: before_script:
- docker info - docker info
build_image: build_image:
script: script:
- docker build -t my-docker-image . - docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests - docker run my-docker-image /script/to/run/tests
``` ```
1. You can now use `docker` command (and **install** `docker-compose` if needed). 1. You can now use `docker` command (and **install** `docker-compose` if needed).
@ -107,83 +107,83 @@ In order to do that, follow the steps:
1. Register GitLab Runner from the command line to use `docker` and `privileged` 1. Register GitLab Runner from the command line to use `docker` and `privileged`
mode: mode:
```bash ```bash
sudo gitlab-runner register -n \ sudo gitlab-runner register -n \
--url https://gitlab.com/ \ --url https://gitlab.com/ \
--registration-token REGISTRATION_TOKEN \ --registration-token REGISTRATION_TOKEN \
--executor docker \ --executor docker \
--description "My Docker Runner" \ --description "My Docker Runner" \
--docker-image "docker:stable" \ --docker-image "docker:stable" \
--docker-privileged --docker-privileged
``` ```
The above command will register a new Runner to use the special The above command will register a new Runner to use the special
`docker:stable` image which is provided by Docker. **Notice that it's using `docker:stable` image which is provided by Docker. **Notice that it's using
the `privileged` mode to start the build and service containers.** If you the `privileged` mode to start the build and service containers.** If you
want to use [docker-in-docker] mode, you always have to use `privileged = true` want to use [docker-in-docker] mode, you always have to use `privileged = true`
in your Docker containers. in your Docker containers.
DANGER: **Danger:** DANGER: **Danger:**
By enabling `--docker-privileged`, you are effectively disabling all of By enabling `--docker-privileged`, you are effectively disabling all of
the security mechanisms of containers and exposing your host to privilege the security mechanisms of containers and exposing your host to privilege
escalation which can lead to container breakout. For more information, check escalation which can lead to container breakout. For more information, check
out the official Docker documentation on out the official Docker documentation on
[Runtime privilege and Linux capabilities][docker-cap]. [Runtime privilege and Linux capabilities][docker-cap].
The above command will create a `config.toml` entry similar to this: The above command will create a `config.toml` entry similar to this:
```toml ```toml
[[runners]] [[runners]]
url = "https://gitlab.com/" url = "https://gitlab.com/"
token = TOKEN token = TOKEN
executor = "docker" executor = "docker"
[runners.docker] [runners.docker]
tls_verify = false tls_verify = false
image = "docker:stable" image = "docker:stable"
privileged = true privileged = true
disable_cache = false disable_cache = false
volumes = ["/cache"] volumes = ["/cache"]
[runners.cache] [runners.cache]
Insecure = false Insecure = false
``` ```
1. You can now use `docker` in the build script (note the inclusion of the 1. You can now use `docker` in the build script (note the inclusion of the
`docker:dind` service): `docker:dind` service):
```yaml ```yaml
image: docker:stable image: docker:stable
variables: variables:
# When using dind service we need to instruct docker, to talk with the # When using dind service we need to instruct docker, to talk with the
# daemon started inside of the service. The daemon is available with # daemon started inside of the service. The daemon is available with
# a network connection instead of the default /var/run/docker.sock socket. # a network connection instead of the default /var/run/docker.sock socket.
# #
# The 'docker' hostname is the alias of the service container as described at # The 'docker' hostname is the alias of the service container as described at
# https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services
# #
# Note that if you're using the Kubernetes executor, the variable should be set to # Note that if you're using the Kubernetes executor, the variable should be set to
# tcp://localhost:2375/ because of how the Kubernetes executor connects services # tcp://localhost:2375/ because of how the Kubernetes executor connects services
# to the job container # to the job container
# DOCKER_HOST: tcp://localhost:2375/ # DOCKER_HOST: tcp://localhost:2375/
# #
# For non-Kubernetes executors, we use tcp://docker:2375/ # For non-Kubernetes executors, we use tcp://docker:2375/
DOCKER_HOST: tcp://docker:2375/ DOCKER_HOST: tcp://docker:2375/
# When using dind, it's wise to use the overlayfs driver for # When using dind, it's wise to use the overlayfs driver for
# improved performance. # improved performance.
DOCKER_DRIVER: overlay2 DOCKER_DRIVER: overlay2
services: services:
- docker:dind - docker:dind
before_script: before_script:
- docker info - docker info
build: build:
stage: build stage: build
script: script:
- docker build -t my-docker-image . - docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests - docker run my-docker-image /script/to/run/tests
``` ```
Docker-in-Docker works well, and is the recommended configuration, but it is Docker-in-Docker works well, and is the recommended configuration, but it is
not without its own challenges: not without its own challenges:
@ -202,14 +202,14 @@ not without its own challenges:
and use it as your mount point (for a more thorough explanation, check [issue and use it as your mount point (for a more thorough explanation, check [issue
#41227](https://gitlab.com/gitlab-org/gitlab-ce/issues/41227)): #41227](https://gitlab.com/gitlab-org/gitlab-ce/issues/41227)):
```yaml ```yaml
variables: variables:
MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt
script: script:
- mkdir -p "$MOUNT_POINT" - mkdir -p "$MOUNT_POINT"
- docker run -v "$MOUNT_POINT:/mnt" my-docker-image - docker run -v "$MOUNT_POINT:/mnt" my-docker-image
``` ```
An example project using this approach can be found here: <https://gitlab.com/gitlab-examples/docker>. An example project using this approach can be found here: <https://gitlab.com/gitlab-examples/docker>.
@ -230,54 +230,54 @@ In order to do that, follow the steps:
1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: 1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`:
```bash ```bash
sudo gitlab-runner register -n \ sudo gitlab-runner register -n \
--url https://gitlab.com/ \ --url https://gitlab.com/ \
--registration-token REGISTRATION_TOKEN \ --registration-token REGISTRATION_TOKEN \
--executor docker \ --executor docker \
--description "My Docker Runner" \ --description "My Docker Runner" \
--docker-image "docker:stable" \ --docker-image "docker:stable" \
--docker-volumes /var/run/docker.sock:/var/run/docker.sock --docker-volumes /var/run/docker.sock:/var/run/docker.sock
``` ```
The above command will register a new Runner to use the special The above command will register a new Runner to use the special
`docker:stable` image which is provided by Docker. **Notice that it's using `docker:stable` image which is provided by Docker. **Notice that it's using
the Docker daemon of the Runner itself, and any containers spawned by docker the Docker daemon of the Runner itself, and any containers spawned by docker
commands will be siblings of the Runner rather than children of the runner.** commands will be siblings of the Runner rather than children of the runner.**
This may have complications and limitations that are unsuitable for your workflow. This may have complications and limitations that are unsuitable for your workflow.
The above command will create a `config.toml` entry similar to this: The above command will create a `config.toml` entry similar to this:
```toml ```toml
[[runners]] [[runners]]
url = "https://gitlab.com/" url = "https://gitlab.com/"
token = REGISTRATION_TOKEN token = REGISTRATION_TOKEN
executor = "docker" executor = "docker"
[runners.docker] [runners.docker]
tls_verify = false tls_verify = false
image = "docker:stable" image = "docker:stable"
privileged = false privileged = false
disable_cache = false disable_cache = false
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
[runners.cache] [runners.cache]
Insecure = false Insecure = false
``` ```
1. You can now use `docker` in the build script (note that you don't need to 1. You can now use `docker` in the build script (note that you don't need to
include the `docker:dind` service as when using the Docker in Docker executor): include the `docker:dind` service as when using the Docker in Docker executor):
```yaml ```yaml
image: docker:stable image: docker:stable
before_script: before_script:
- docker info - docker info
build: build:
stage: build stage: build
script: script:
- docker build -t my-docker-image . - docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests - docker run my-docker-image /script/to/run/tests
``` ```
While the above method avoids using Docker in privileged mode, you should be While the above method avoids using Docker in privileged mode, you should be
aware of the following implications: aware of the following implications:
@ -293,9 +293,9 @@ aware of the following implications:
work as expected since volume mounting is done in the context of the host work as expected since volume mounting is done in the context of the host
machine, not the build container. For example: machine, not the build container. For example:
```sh ```sh
docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests
``` ```
## Making docker-in-docker builds faster with Docker layer caching ## Making docker-in-docker builds faster with Docker layer caching
@ -366,23 +366,23 @@ which can be avoided if a different driver is used, for example `overlay2`.
1. Make sure a recent kernel is used, preferably `>= 4.2`. 1. Make sure a recent kernel is used, preferably `>= 4.2`.
1. Check whether the `overlay` module is loaded: 1. Check whether the `overlay` module is loaded:
```sh ```sh
sudo lsmod | grep overlay sudo lsmod | grep overlay
``` ```
If you see no result, then it isn't loaded. To load it use: If you see no result, then it isn't loaded. To load it use:
```sh ```sh
sudo modprobe overlay sudo modprobe overlay
``` ```
If everything went fine, you need to make sure module is loaded on reboot. If everything went fine, you need to make sure module is loaded on reboot.
On Ubuntu systems, this is done by editing `/etc/modules`. Just add the On Ubuntu systems, this is done by editing `/etc/modules`. Just add the
following line into it: following line into it:
```text ```text
overlay overlay
``` ```
### Use driver per project ### Use driver per project
@ -450,9 +450,9 @@ For all projects, mostly suitable for public ones:
your Docker images and has read/write access to the Registry. This is ephemeral, your Docker images and has read/write access to the Registry. This is ephemeral,
so it's only valid for one job. You can use the following example as-is: so it's only valid for one job. You can use the following example as-is:
```sh ```sh
docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
``` ```
For private and internal projects: For private and internal projects:
@ -465,9 +465,9 @@ For private and internal projects:
Replace the `<username>` and `<access_token>` in the following example: Replace the `<username>` and `<access_token>` in the following example:
```sh ```sh
docker login -u <username> -p <access_token> $CI_REGISTRY docker login -u <username> -p <access_token> $CI_REGISTRY
``` ```
- **Using the GitLab Deploy Token**: You can create and use a - **Using the GitLab Deploy Token**: You can create and use a
[special deploy token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token) [special deploy token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token)
@ -475,9 +475,9 @@ For private and internal projects:
Once created, you can use the special environment variables, and GitLab CI/CD Once created, you can use the special environment variables, and GitLab CI/CD
will fill them in for you. You can use the following example as-is: will fill them in for you. You can use the following example as-is:
```sh ```sh
docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY
``` ```
### Container Registry examples ### Container Registry examples

131
doc/ci/docker/using_docker_images.md
View File

@ -305,25 +305,25 @@ For example, the following two definitions are equal:
1. Using a string as an option to `image` and `services`: 1. Using a string as an option to `image` and `services`:
```yaml ```yaml
image: "registry.example.com/my/image:latest" image: "registry.example.com/my/image:latest"
services: services:
- postgresql:9.4 - postgresql:9.4
- redis:latest - redis:latest
``` ```
1. Using a map as an option to `image` and `services`. The use of `image:name` is 1. Using a map as an option to `image` and `services`. The use of `image:name` is
required: required:
```yaml ```yaml
image: image:
name: "registry.example.com/my/image:latest" name: "registry.example.com/my/image:latest"
services: services:
- name: postgresql:9.4 - name: postgresql:9.4
- name: redis:latest - name: redis:latest
``` ```
### Available settings for `image` ### Available settings for `image`
@ -526,6 +526,7 @@ it's provided as an environment variable. This is because GitLab Runnner uses **
runtime. runtime.
### Using statically-defined credentials ### Using statically-defined credentials
There are two approaches that you can take in order to access a There are two approaches that you can take in order to access a
private registry. Both require setting the environment variable private registry. Both require setting the environment variable
`DOCKER_AUTH_CONFIG` with appropriate authentication info. `DOCKER_AUTH_CONFIG` with appropriate authentication info.
@ -555,18 +556,18 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`:
- **First way -** Do a `docker login` on your local machine: - **First way -** Do a `docker login` on your local machine:
```bash ```bash
docker login registry.example.com:5000 --username my_username --password my_password docker login registry.example.com:5000 --username my_username --password my_password
``` ```
Then copy the content of `~/.docker/config.json`. Then copy the content of `~/.docker/config.json`.
If you don't need access to the registry from your computer, you If you don't need access to the registry from your computer, you
can do a `docker logout`: can do a `docker logout`:
```bash ```bash
docker logout registry.example.com:5000 docker logout registry.example.com:5000
``` ```
- **Second way -** In some setups, it's possible that Docker client - **Second way -** In some setups, it's possible that Docker client
will use the available system keystore to store the result of `docker will use the available system keystore to store the result of `docker
@ -575,12 +576,12 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`:
`${username}:${password}` manually. Open a terminal and execute the `${username}:${password}` manually. Open a terminal and execute the
following command: following command:
```bash ```bash
echo -n "my_username:my_password" | base64 echo -n "my_username:my_password" | base64
# Example output to copy # Example output to copy
bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=
``` ```
#### Configuring a job #### Configuring a job
@ -590,25 +591,25 @@ follow these steps:
1. Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the 1. Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the
Docker configuration file as the value: Docker configuration file as the value:
```json ```json
{ {
"auths": { "auths": {
"registry.example.com:5000": { "registry.example.com:5000": {
"auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=" "auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ="
} }
} }
} }
``` ```
1. You can now use any private image from `registry.example.com:5000` defined in 1. You can now use any private image from `registry.example.com:5000` defined in
`image` and/or `services` in your `.gitlab-ci.yml` file: `image` and/or `services` in your `.gitlab-ci.yml` file:
```yaml ```yaml
image: registry.example.com:5000/namespace/image:tag image: registry.example.com:5000/namespace/image:tag
``` ```
In the example above, GitLab Runner will look at `registry.example.com:5000` for the In the example above, GitLab Runner will look at `registry.example.com:5000` for the
image `namespace/image:tag`. image `namespace/image:tag`.
You can add configuration for as many registries as you want, adding more You can add configuration for as many registries as you want, adding more
registries to the `"auths"` hash as described above. registries to the `"auths"` hash as described above.
@ -637,10 +638,10 @@ To add `DOCKER_AUTH_CONFIG` to a Runner:
1. Modify the Runner's `config.toml` file as follows: 1. Modify the Runner's `config.toml` file as follows:
```toml ```toml
[[runners]] [[runners]]
environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"registry.example.com:5000\":{\"auth\":\"bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=\"}}}"] environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"registry.example.com:5000\":{\"auth\":\"bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=\"}}}"]
``` ```
1. Restart the Runner service. 1. Restart the Runner service.
@ -662,16 +663,17 @@ To configure credentials store, follow these steps:
Make sure helper program is available in GitLab Runner `$PATH`. Make sure helper program is available in GitLab Runner `$PATH`.
1. Make GitLab Runner use it. There are two ways to accomplish this. Either: 1. Make GitLab Runner use it. There are two ways to accomplish this. Either:
- Create a - Create a
[variable](../variables/README.md#gitlab-cicd-environment-variables) [variable](../variables/README.md#gitlab-cicd-environment-variables)
`DOCKER_AUTH_CONFIG` with the content of the `DOCKER_AUTH_CONFIG` with the content of the
Docker configuration file as the value: Docker configuration file as the value:
```json ```json
{ {
"credsStore": "osxkeychain" "credsStore": "osxkeychain"
} }
``` ```
- Or, if you are running self-hosted Runners, add the above JSON to - Or, if you are running self-hosted Runners, add the above JSON to
`${GITLAB_RUNNER_HOME}/.docker/config.json`. GitLab Runner will read this config file `${GITLAB_RUNNER_HOME}/.docker/config.json`. GitLab Runner will read this config file
@ -693,17 +695,18 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th
1. Make sure `docker-credential-ecr-login` is available in GitLab Runner's `$PATH`. 1. Make sure `docker-credential-ecr-login` is available in GitLab Runner's `$PATH`.
1. Make GitLab Runner use it. There are two ways to accomplish this. Either: 1. Make GitLab Runner use it. There are two ways to accomplish this. Either:
- Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) - Create a [variable](../variables/README.md#gitlab-cicd-environment-variables)
`DOCKER_AUTH_CONFIG` with the content of the `DOCKER_AUTH_CONFIG` with the content of the
Docker configuration file as the value: Docker configuration file as the value:
```json ```json
{ {
"credHelpers": { "credHelpers": {
"aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login" "aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login"
} }
} }
``` ```
- Or, if you are running self-hosted Runners, - Or, if you are running self-hosted Runners,
add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`. add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`.
@ -713,12 +716,12 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th
1. You can now use any private image from `aws_account_id.dkr.ecr.region.amazonaws.com` defined in 1. You can now use any private image from `aws_account_id.dkr.ecr.region.amazonaws.com` defined in
`image` and/or `services` in your `.gitlab-ci.yml` file: `image` and/or `services` in your `.gitlab-ci.yml` file:
```yaml ```yaml
image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest
``` ```
In the example above, GitLab Runner will look at `aws_account_id.dkr.ecr.region.amazonaws.com` for the In the example above, GitLab Runner will look at `aws_account_id.dkr.ecr.region.amazonaws.com` for the
image `private/image:latest`. image `private/image:latest`.
You can add configuration for as many registries as you want, adding more You can add configuration for as many registries as you want, adding more
registries to the `"credHelpers"` hash as described above. registries to the `"credHelpers"` hash as described above.

70
doc/ci/examples/artifactory_and_gitlab/index.md
View File

@ -39,9 +39,10 @@ project:
1. Create a new project by selecting **Import project from ➔ Repo by URL** 1. Create a new project by selecting **Import project from ➔ Repo by URL**
1. Add the following URL: 1. Add the following URL:
``` ```
https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git
``` ```
1. Click **Create project** 1. Click **Create project**
This application is nothing more than a basic class with a stub for a JUnit based test suite. This application is nothing more than a basic class with a stub for a JUnit based test suite.
@ -63,15 +64,15 @@ The application is ready to use, but you need some additional steps to deploy it
1. Copy the snippet in the `pom.xml` file for your project, just after the 1. Copy the snippet in the `pom.xml` file for your project, just after the
`dependencies` section. The snippet should look like this: `dependencies` section. The snippet should look like this:
```xml ```xml
<distributionManagement> <distributionManagement>
<repository> <repository>
<id>central</id> <id>central</id>
<name>83d43b5afeb5-releases</name> <name>83d43b5afeb5-releases</name>
<url>${env.MAVEN_REPO_URL}/libs-release-local</url> <url>${env.MAVEN_REPO_URL}/libs-release-local</url>
</repository> </repository>
</distributionManagement> </distributionManagement>
``` ```
Another step you need to do before you can deploy the dependency to Artifactory Another step you need to do before you can deploy the dependency to Artifactory
is to configure the authentication data. It is a simple task, but Maven requires is to configure the authentication data. It is a simple task, but Maven requires
@ -86,18 +87,18 @@ parameter in `.gitlab-ci.yml` to use the custom location instead of the default
1. Create a file called `settings.xml` in the `.m2` folder 1. Create a file called `settings.xml` in the `.m2` folder
1. Copy the following content into a `settings.xml` file: 1. Copy the following content into a `settings.xml` file:
```xml ```xml
<settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd"
xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<servers> <servers>
<server> <server>
<id>central</id> <id>central</id>
<username>${env.MAVEN_REPO_USER}</username> <username>${env.MAVEN_REPO_USER}</username>
<password>${env.MAVEN_REPO_PASS}</password> <password>${env.MAVEN_REPO_PASS}</password>
</server> </server>
</servers> </servers>
</settings> </settings>
``` ```
Username and password will be replaced by the correct values using variables. Username and password will be replaced by the correct values using variables.
@ -187,9 +188,10 @@ We'll use again a Maven app that can be cloned from our example project:
1. Create a new project by selecting **Import project from ➔ Repo by URL** 1. Create a new project by selecting **Import project from ➔ Repo by URL**
1. Add the following URL: 1. Add the following URL:
``` ```
https://gitlab.com/gitlab-examples/maven/simple-maven-app.git https://gitlab.com/gitlab-examples/maven/simple-maven-app.git
``` ```
1. Click **Create project** 1. Click **Create project**
This one is a simple app as well. If you look at the `src/main/java/com/example/app/App.java` This one is a simple app as well. If you look at the `src/main/java/com/example/app/App.java`
@ -204,13 +206,13 @@ Since Maven doesn't know how to resolve the dependency, you need to modify the c
1. Copy the snippet in the `dependencies` section of the `pom.xml` file. 1. Copy the snippet in the `dependencies` section of the `pom.xml` file.
The snippet should look like this: The snippet should look like this:
```xml ```xml
<dependency> <dependency>
<groupId>com.example.dep</groupId> <groupId>com.example.dep</groupId>
<artifactId>simple-maven-dep</artifactId> <artifactId>simple-maven-dep</artifactId>
<version>1.0</version> <version>1.0</version>
</dependency> </dependency>
``` ```
### Configure the Artifactory repository location ### Configure the Artifactory repository location

107
doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md
View File

@ -188,28 +188,27 @@ when running our Phoenix in our `localhost`.
- Open `hello_gitlab_ci/config/test.exs` on your favorite code editor - Open `hello_gitlab_ci/config/test.exs` on your favorite code editor
- Go to **Configure your database** session and edit the block to include `System.get_env`: - Go to **Configure your database** session and edit the block to include `System.get_env`:
```elixir ```elixir
# Configure your database # Configure your database
config :hello_gitlab_ci, HelloGitlabCi.Repo, config :hello_gitlab_ci, HelloGitlabCi.Repo,
adapter: Ecto.Adapters.Postgres, adapter: Ecto.Adapters.Postgres,
username: System.get_env("POSTGRES_USER") || "postgres", username: System.get_env("POSTGRES_USER") || "postgres",
password: System.get_env("POSTGRES_PASSWORD") || "postgres", password: System.get_env("POSTGRES_PASSWORD") || "postgres",
database: System.get_env("POSTGRES_DB") || "hello_gitlab_ci_test", database: System.get_env("POSTGRES_DB") || "hello_gitlab_ci_test",
hostname: System.get_env("POSTGRES_HOST") || "localhost", hostname: System.get_env("POSTGRES_HOST") || "localhost",
pool: Ecto.Adapters.SQL.Sandbox pool: Ecto.Adapters.SQL.Sandbox
``` ```
We'll need these system variables later on. We'll need these system variables later on.
- Create an empty file named `.gitkeep` into `hello_gitlab_ci/priv/repo/migrations` - Create an empty file named `.gitkeep` into `hello_gitlab_ci/priv/repo/migrations`
As our project is still fresh, we don't have any data on our database, so, the `migrations` As our project is still fresh, we don't have any data on our database, so, the `migrations`
directory will be empty. directory will be empty.
Without `.gitkeep`, git will not upload this empty directory and we'll got an error when running our Without `.gitkeep`, git will not upload this empty directory and we'll got an error when running our
test on GitLab. test on GitLab.
> **Note:** > **Note:** If we add a folder via the GitLab UI, GitLab itself will add the `.gitkeep` to that new dir.
If we add a folder via the GitLab UI, GitLab itself will add the `.gitkeep` to that new dir.
Now, let's run a local test and see if everything we did didn't break anything. Now, let's run a local test and see if everything we did didn't break anything.
@ -248,64 +247,64 @@ project.
- The fastest and easiest way to do this, is to click on **Set up CI** on project's main page: - The fastest and easiest way to do this, is to click on **Set up CI** on project's main page:
![Set up CI](img/setup-ci.png) ![Set up CI](img/setup-ci.png)
- On next screen, we can select a template ready to go. Click on **Apply a GitLab CI/CD Yaml - On next screen, we can select a template ready to go. Click on **Apply a GitLab CI/CD Yaml
template** and select **Elixir**: template** and select **Elixir**:
![Select template](img/select-template.png) ![Select template](img/select-template.png)
This template file tells GitLab CI/CD about what we wish to do every time a new commit is made. This template file tells GitLab CI/CD about what we wish to do every time a new commit is made.
However, we have to adapt it to run a Phoenix app. However, we have to adapt it to run a Phoenix app.
- The first line tells GitLab what Docker image will be used. - The first line tells GitLab what Docker image will be used.
Remember when we learn about Runners, the isolated virtual machine where GitLab CI/CD build and test Remember when we learn about Runners, the isolated virtual machine where GitLab CI/CD build and test
our application? This virtual machine must have all dependencies to run our application. This is our application? This virtual machine must have all dependencies to run our application. This is
where a Docker image is needed. The correct image will provide the entire system for us. where a Docker image is needed. The correct image will provide the entire system for us.
As a suggestion, you can use [trenpixster's elixir image][docker-image], which already has all As a suggestion, you can use [trenpixster's elixir image][docker-image], which already has all
dependencies for Phoenix installed, such as Elixir, Erlang, NodeJS and PostgreSQL: dependencies for Phoenix installed, such as Elixir, Erlang, NodeJS and PostgreSQL:
```yml ```yml
image: trenpixster/elixir:latest image: trenpixster/elixir:latest
``` ```
- At `services` session, we'll only use `postgres`, so we'll delete `mysql` and `redis` lines: - At `services` session, we'll only use `postgres`, so we'll delete `mysql` and `redis` lines:
```yml ```yml
services: services:
- postgres:latest - postgres:latest
``` ```
- Now, we'll create a new entry called `variables`, before `before_script` session: - Now, we'll create a new entry called `variables`, before `before_script` session:
```yml ```yml
variables: variables:
POSTGRES_DB: hello_gitlab_ci_test POSTGRES_DB: hello_gitlab_ci_test
POSTGRES_HOST: postgres POSTGRES_HOST: postgres
POSTGRES_USER: postgres POSTGRES_USER: postgres
POSTGRES_PASSWORD: "postgres" POSTGRES_PASSWORD: "postgres"
MIX_ENV: "test" MIX_ENV: "test"
``` ```
Here, we are setting up the values for GitLab CI/CD authenticate into PostgreSQL, as we did on Here, we are setting up the values for GitLab CI/CD authenticate into PostgreSQL, as we did on
`config/test.exs` earlier. `config/test.exs` earlier.
- In `before_script` session, we'll add some commands to prepare everything to the test: - In `before_script` session, we'll add some commands to prepare everything to the test:
```yml ```yml
before_script: before_script:
- apt-get update && apt-get -y install postgresql-client - apt-get update && apt-get -y install postgresql-client
- mix local.hex --force - mix local.hex --force
- mix deps.get --only test - mix deps.get --only test
- mix ecto.create - mix ecto.create
- mix ecto.migrate - mix ecto.migrate
``` ```
It's important to install `postgresql-client` to let GitLab CI/CD access PostgreSQL and create our It's important to install `postgresql-client` to let GitLab CI/CD access PostgreSQL and create our
database with the login information provided earlier. More important is to respect the indentation, database with the login information provided earlier. More important is to respect the indentation,
to avoid syntax errors when running the build. to avoid syntax errors when running the build.
- And finally, we'll let `mix` session intact. - And finally, we'll let `mix` session intact.

72
doc/ci/ssh_keys/README.md
View File

@ -57,44 +57,44 @@ to access it. This is where an SSH key pair comes in handy.
1. Modify your `.gitlab-ci.yml` with a `before_script` action. In the following 1. Modify your `.gitlab-ci.yml` with a `before_script` action. In the following
example, a Debian based image is assumed. Edit to your needs: example, a Debian based image is assumed. Edit to your needs:
```yaml ```yaml
before_script: before_script:
## ##
## Install ssh-agent if not already installed, it is required by Docker. ## Install ssh-agent if not already installed, it is required by Docker.
## (change apt-get to yum if you use an RPM-based image) ## (change apt-get to yum if you use an RPM-based image)
## ##
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
## ##
## Run ssh-agent (inside the build environment) ## Run ssh-agent (inside the build environment)
## ##
- eval $(ssh-agent -s) - eval $(ssh-agent -s)
## ##
## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
## We're using tr to fix line endings which makes ed25519 keys work ## We're using tr to fix line endings which makes ed25519 keys work
## without extra base64 encoding. ## without extra base64 encoding.
## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556 ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
## ##
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
## ##
## Create the SSH directory and give it the right permissions ## Create the SSH directory and give it the right permissions
## ##
- mkdir -p ~/.ssh - mkdir -p ~/.ssh
- chmod 700 ~/.ssh - chmod 700 ~/.ssh
## ##
## Optionally, if you will be using any Git commands, set the user name and ## Optionally, if you will be using any Git commands, set the user name and
## and email. ## and email.
## ##
#- git config --global user.email "user@example.com" #- git config --global user.email "user@example.com"
#- git config --global user.name "User name" #- git config --global user.name "User name"
``` ```
NOTE: **Note:** NOTE: **Note:**
The [`before_script`](../yaml/README.md#before_script-and-after_script) can be set globally The [`before_script`](../yaml/README.md#before_script-and-after_script) can be set globally
or per-job. or per-job.
1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys). 1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys).
@ -118,9 +118,9 @@ on, and use that key for all projects that are run on this machine.
1. Then from the terminal login as the `gitlab-runner` user: 1. Then from the terminal login as the `gitlab-runner` user:
``` ```
sudo su - gitlab-runner sudo su - gitlab-runner
``` ```
1. Generate the SSH key pair as described in the instructions to 1. Generate the SSH key pair as described in the instructions to
[generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair). [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair).

16
doc/ci/yaml/README.md
View File

@ -2468,20 +2468,20 @@ There are three possible values: `none`, `normal`, and `recursive`:
- `normal` means that only the top-level submodules will be included. It is - `normal` means that only the top-level submodules will be included. It is
equivalent to: equivalent to:
``` ```
git submodule sync git submodule sync
git submodule update --init git submodule update --init
``` ```
- `recursive` means that all submodules (including submodules of submodules) - `recursive` means that all submodules (including submodules of submodules)
will be included. This feature needs Git v1.8.1 and later. When using a will be included. This feature needs Git v1.8.1 and later. When using a
GitLab Runner with an executor not based on Docker, make sure the Git version GitLab Runner with an executor not based on Docker, make sure the Git version
meets that requirement. It is equivalent to: meets that requirement. It is equivalent to:
``` ```
git submodule sync --recursive git submodule sync --recursive
git submodule update --init --recursive git submodule update --init --recursive
``` ```
Note that for this feature to work correctly, the submodules must be configured Note that for this feature to work correctly, the submodules must be configured
(in `.gitmodules`) with either: (in `.gitmodules`) with either: