kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Make usage of SSH keys in CI/CD more clear

Browse source
This commit is contained in:
Achilleas Pipinellis 2017-12-07 15:29:40 +01:00
parent a11fae2489
commit f06982bf92
No known key found for this signature in database
GPG key ID: A0996FBD3E92C17B
3 changed files with 166 additions and 76 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

221
doc/ci/ssh_keys/README.md
View file

@ -1,84 +1,106 @@
# Using SSH keys ---
last_updated: 2017-12-13
---
# Using SSH keys with GitLab CI/CD
GitLab currently doesn't have built-in support for managing SSH keys in a build GitLab currently doesn't have built-in support for managing SSH keys in a build
environment. environment (where the GitLab Runner runs).
The SSH keys can be useful when: The SSH keys can be useful when:
1. You want to checkout internal submodules 1. You want to checkout internal submodules
2. You want to download private packages using your package manager (eg. bundler) 1. You want to download private packages using your package manager (e.g., Bundler)
3. You want to deploy your application to eg. Heroku or your own server 1. You want to deploy your application to your own server, or, for example, Heroku
4. You want to execute SSH commands from the build server to the remote server 1. You want to execute SSH commands from the build environment to a remote server
5. You want to rsync files from your build server to the remote server 1. You want to rsync files from the build environment to a remote server
If anything of the above rings a bell, then you most likely need an SSH key. If anything of the above rings a bell, then you most likely need an SSH key.
## Inject keys in your build server
The most widely supported method is to inject an SSH key into your build The most widely supported method is to inject an SSH key into your build
environment by extending your `.gitlab-ci.yml`. environment by extending your `.gitlab-ci.yml`, and it's a solution which works
with any type of [executor](https://docs.gitlab.com/runner/executors/)
(Docker, shell, etc.).
This is the universal solution which works with any type of executor ## How it works
(docker, shell, etc.).
### How it works 1. Create a new SSH key pair locally with [ssh-keygen](http://linux.die.net/man/1/ssh-keygen)
1. Add the private key as a [secret variable](../variables/README.md) to
your project
1. Run the [ssh-agent](http://linux.die.net/man/1/ssh-agent) during job to load
the private key.
1. Copy the public key to the servers you want to have access to (usually in
`~/.ssh/authorized_keys`) or add it as a [deploy key](../../ssh/README.md#deploy-keys)
if you are accessing a private GitLab repository.
1. Create a new SSH key pair with [ssh-keygen][] NOTE: **Note:**
2. Add the private key as a **Secret Variable** to the project The private key will not be displayed in the job trace, unless you enable
3. Run the [ssh-agent][] during job to load the private key. [debug tracing](../variables/README.md#debug-tracing). You might also want to
check the [visibility of your pipelines](../../user/project/pipelines/settings.md#visibility-of-pipelines).
## SSH keys when using the Docker executor ## SSH keys when using the Docker executor
You will first need to create an SSH key pair. For more information, follow the When your CI/CD jobs run inside Docker containers (meaning the environment is
instructions to [generate an SSH key](../../ssh/README.md). Do not add a contained) and you want to deploy your code in a private server, you need a way
passphrase to the SSH key, or the `before_script` will prompt for it. to access it. This is where an SSH key pair comes in handy.
Then, create a new **Secret Variable** in your project settings on GitLab 1. You will first need to create an SSH key pair. For more information, follow
following **Settings > CI/CD** and look for the "Secret Variables" section. the instructions to [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair).
As **Key** add the name `SSH_PRIVATE_KEY` and in the **Value** field paste the **Do not** add a passphrase to the SSH key, or the `before_script` will\
content of your _private_ key that you created earlier. prompt for it.
It is also good practice to check the server's own public key to make sure you 1. Create a new [secret variable](../variables/README.md#secret-variables).
are not being targeted by a man-in-the-middle attack. To do this, add another As **Key** enter the name `SSH_PRIVATE_KEY` and in the **Value** field paste
variable named `SSH_SERVER_HOSTKEYS`. To find out the hostkeys of your server, run the content of your _private_ key that you created earlier.
the `ssh-keyscan YOUR_SERVER` command from a trusted network (ideally, from the
server itself), and paste its output into the `SSH_SERVER_HOSTKEYS` variable. If
you need to connect to multiple servers, concatenate all the server public keys
that you collected into the **Value** of the variable. There must be one key per
line.
Next you need to modify your `.gitlab-ci.yml` with a `before_script` action. 1. Modify your `.gitlab-ci.yml` with a `before_script` action. In the following
Add it to the top: example, a Debian based image is assumed. Edit to your needs:
``` ```yaml
before_script: before_script:
# Install ssh-agent if not already installed, it is required by Docker. ##
# (change apt-get to yum if you use a CentOS-based image) ## Install ssh-agent if not already installed, it is required by Docker.
## (change apt-get to yum if you use an RPM-based image)
##
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# Run ssh-agent (inside the build environment) ##
## Run ssh-agent (inside the build environment)
##
- eval $(ssh-agent -s) - eval $(ssh-agent -s)
# Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store ##
- ssh-add <(echo "$SSH_PRIVATE_KEY") ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
## We're using tr to fix line endings which makes ed25519 keys work
## without extra base64 encoding.
## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
##
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
# For Docker builds disable host key checking. Be aware that by adding that ##
# you are suspectible to man-in-the-middle attacks. ## Create the SSH directory and give it the right permissions
# WARNING: Use this only with the Docker executor, if you use it with shell ##
# you will overwrite your user's SSH config.
- mkdir -p ~/.ssh - mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' - chmod 700 ~/.ssh
# In order to properly check the server's host key, assuming you created the
# SSH_SERVER_HOSTKEYS variable previously, uncomment the following two lines
# instead.
# - mkdir -p ~/.ssh
# - '[[ -f /.dockerenv ]] && echo "$SSH_SERVER_HOSTKEYS" > ~/.ssh/known_hosts'
```
As a final step, add the _public_ key from the one you created earlier to the ##
services that you want to have an access to from within the build environment. ## Optionally, if you will be using any Git commands, set the user name and
If you are accessing a private GitLab repository you need to add it as a ## and email.
[deploy key](../../ssh/README.md#deploy-keys). ##
#- git config --global user.email "user@example.com"
#- git config --global user.name "User name"
```
NOTE: **Note:**
The [`before_script`](../yaml/README.md#before-script) can be set globally
or per-job.
1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys).
1. As a final step, add the _public_ key from the one you created in the first
step to the services that you want to have an access to from within the build
environment. If you are accessing a private GitLab repository you need to add
it as a [deploy key](../../ssh/README.md#deploy-keys).
That's it! You can now have access to private servers or repositories in your That's it! You can now have access to private servers or repositories in your
build environment. build environment.
@ -91,24 +113,93 @@ SSH key.
You can generate the SSH key from the machine that GitLab Runner is installed You can generate the SSH key from the machine that GitLab Runner is installed
on, and use that key for all projects that are run on this machine. on, and use that key for all projects that are run on this machine.
First, you need to login to the server that runs your jobs. 1. First, you need to login to the server that runs your jobs.
Then from the terminal login as the `gitlab-runner` user and generate the SSH 1. Then from the terminal login as the `gitlab-runner` user:
key pair as described in the [SSH keys documentation](../../ssh/README.md).
As a final step, add the _public_ key from the one you created earlier to the ```
services that you want to have an access to from within the build environment. sudo su - gitlab-runner
If you are accessing a private GitLab repository you need to add it as a ```
[deploy key](../../ssh/README.md#deploy-keys).
1. Generate the SSH key pair as described in the instructions to
[generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair).
**Do not** add a passphrase to the SSH key, or the `before_script` will
prompt for it.
1. As a final step, add the _public_ key from the one you created earlier to the
services that you want to have an access to from within the build environment.
If you are accessing a private GitLab repository you need to add it as a
[deploy key](../../ssh/README.md#deploy-keys).
Once done, try to login to the remote server in order to accept the fingerprint: Once done, try to login to the remote server in order to accept the fingerprint:
```bash ```bash
ssh <address-of-my-server> ssh example.com
``` ```
For accessing repositories on GitLab.com, the `<address-of-my-server>` would be For accessing repositories on GitLab.com, you would use `git@gitlab.com`.
`git@gitlab.com`.
## Verifying the SSH host keys
It is a good practice to check the private server's own public key to make sure
you are not being targeted by a man-in-the-middle attack. In case anything
suspicious happens, you will notice it since the job would fail (the SSH
connection would fail if the public keys would not match).
To find out the host keys of your server, run the `ssh-keyscan` command from a
trusted network (ideally, from the private server itself):
```sh
## Use the domain name
ssh-keyscan example.com
## Or use an IP
ssh-keyscan 1.2.3.4
```
Create a new [secret variable](../variables/README.md#secret-variables) with
`SSH_KNOWN_HOSTS` as "Key", and as a "Value" add the output of `ssh-keyscan`.
NOTE: **Note:**
If you need to connect to multiple servers, all the server host keys
need to be collected in the **Value** of the variable, one key per line.
TIP: **Tip:**
By using a secret variable instead of `ssh-keyscan` directly inside
`.gitlab-ci.yml`, it has the benefit that you don't have to change `.gitlab-ci.yml`
if the host domain name changes for some reason. Also, the values are predefined
by you, meaning that if the host keys suddenly change, the CI/CD job will fail,
and you'll know there's something wrong with the server or the network.
Now that the `SSH_KNOWN_HOSTS` variable is created, in addition to the
[content of `.gitlab-ci.yml`](#ssh-keys-when-using-the-docker-executor)
above, here's what more you need to add:
```yaml
before_script:
##
## Assuming you created the SSH_KNOWN_HOSTS variable, uncomment the
## following two lines.
##
- echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts'
- chmod 644 ~/.ssh/known_hosts
##
## Alternatively, use ssh-keyscan to scan the keys of your private server.
## Replace example.com with your private server's domain name. Repeat that
## command if you have more than one server to connect to.
##
#- ssh-keyscan example.com >> ~/.ssh/known_hosts
#- chmod 644 ~/.ssh/known_hosts
##
## You can optionally disable host key checking. Be aware that by adding that
## you are suspectible to man-in-the-middle attacks.
## WARNING: Use this only with the Docker executor, if you use it with shell
## you will overwrite your user's SSH config.
##
#- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
```
## Example project ## Example project
@ -119,6 +210,4 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available
Want to hack on it? Simply fork it, commit and push your changes. Within a few Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the job will begin. moments the changes will be picked by a public runner and the job will begin.
[ssh-keygen]: http://linux.die.net/man/1/ssh-keygen
[ssh-agent]: http://linux.die.net/man/1/ssh-agent
[ssh-example-repo]: https://gitlab.com/gitlab-examples/ssh-private-key/ [ssh-example-repo]: https://gitlab.com/gitlab-examples/ssh-private-key/

13
doc/ci/variables/README.md
View file

@ -213,14 +213,15 @@ An example project service that defines deployment variables is
## Debug tracing ## Debug tracing
> Introduced in GitLab Runner 1.7. > Introduced in GitLab Runner 1.7.
>
> **WARNING:** Enabling debug tracing can have severe security implications. The CAUTION: **Warning:**
output **will** contain the content of all your secret variables and any other Enabling debug tracing can have severe security implications. The
secrets! The output **will** be uploaded to the GitLab server and made visible output **will** contain the content of all your secret variables and any other
in job traces! secrets! The output **will** be uploaded to the GitLab server and made visible
in job traces!
By default, GitLab Runner hides most of the details of what it is doing when By default, GitLab Runner hides most of the details of what it is doing when
processing a job. This behaviour keeps job traces short, and prevents secrets processing a job. This behavior keeps job traces short, and prevents secrets
from being leaked into the trace unless your script writes them to the screen. from being leaked into the trace unless your script writes them to the screen.
If a job isn't working as expected, this can make the problem difficult to If a job isn't working as expected, this can make the problem difficult to

2
doc/user/project/pipelines/settings.md
View file

@ -68,7 +68,7 @@ in the pipelines settings page.
Access to pipelines and job details (including output of logs and artifacts) Access to pipelines and job details (including output of logs and artifacts)
is checked against your current user access level and the **Public pipelines** is checked against your current user access level and the **Public pipelines**
project setting. project setting under your project's **Settings > CI/CD > General pipelines settings**.
If **Public pipelines** is enabled (default): If **Public pipelines** is enabled (default):