Make usage of SSH keys in CI/CD more clear
This commit is contained in:
parent
a11fae2489
commit
f06982bf92
3 changed files with 166 additions and 76 deletions
|
@ -1,84 +1,106 @@
|
|||
# Using SSH keys
|
||||
---
|
||||
last_updated: 2017-12-13
|
||||
---
|
||||
|
||||
# Using SSH keys with GitLab CI/CD
|
||||
|
||||
GitLab currently doesn't have built-in support for managing SSH keys in a build
|
||||
environment.
|
||||
environment (where the GitLab Runner runs).
|
||||
|
||||
The SSH keys can be useful when:
|
||||
|
||||
1. You want to checkout internal submodules
|
||||
2. You want to download private packages using your package manager (eg. bundler)
|
||||
3. You want to deploy your application to eg. Heroku or your own server
|
||||
4. You want to execute SSH commands from the build server to the remote server
|
||||
5. You want to rsync files from your build server to the remote server
|
||||
1. You want to download private packages using your package manager (e.g., Bundler)
|
||||
1. You want to deploy your application to your own server, or, for example, Heroku
|
||||
1. You want to execute SSH commands from the build environment to a remote server
|
||||
1. You want to rsync files from the build environment to a remote server
|
||||
|
||||
If anything of the above rings a bell, then you most likely need an SSH key.
|
||||
|
||||
## Inject keys in your build server
|
||||
|
||||
The most widely supported method is to inject an SSH key into your build
|
||||
environment by extending your `.gitlab-ci.yml`.
|
||||
environment by extending your `.gitlab-ci.yml`, and it's a solution which works
|
||||
with any type of [executor](https://docs.gitlab.com/runner/executors/)
|
||||
(Docker, shell, etc.).
|
||||
|
||||
This is the universal solution which works with any type of executor
|
||||
(docker, shell, etc.).
|
||||
## How it works
|
||||
|
||||
### How it works
|
||||
1. Create a new SSH key pair locally with [ssh-keygen](http://linux.die.net/man/1/ssh-keygen)
|
||||
1. Add the private key as a [secret variable](../variables/README.md) to
|
||||
your project
|
||||
1. Run the [ssh-agent](http://linux.die.net/man/1/ssh-agent) during job to load
|
||||
the private key.
|
||||
1. Copy the public key to the servers you want to have access to (usually in
|
||||
`~/.ssh/authorized_keys`) or add it as a [deploy key](../../ssh/README.md#deploy-keys)
|
||||
if you are accessing a private GitLab repository.
|
||||
|
||||
1. Create a new SSH key pair with [ssh-keygen][]
|
||||
2. Add the private key as a **Secret Variable** to the project
|
||||
3. Run the [ssh-agent][] during job to load the private key.
|
||||
NOTE: **Note:**
|
||||
The private key will not be displayed in the job trace, unless you enable
|
||||
[debug tracing](../variables/README.md#debug-tracing). You might also want to
|
||||
check the [visibility of your pipelines](../../user/project/pipelines/settings.md#visibility-of-pipelines).
|
||||
|
||||
## SSH keys when using the Docker executor
|
||||
|
||||
You will first need to create an SSH key pair. For more information, follow the
|
||||
instructions to [generate an SSH key](../../ssh/README.md). Do not add a
|
||||
passphrase to the SSH key, or the `before_script` will prompt for it.
|
||||
When your CI/CD jobs run inside Docker containers (meaning the environment is
|
||||
contained) and you want to deploy your code in a private server, you need a way
|
||||
to access it. This is where an SSH key pair comes in handy.
|
||||
|
||||
Then, create a new **Secret Variable** in your project settings on GitLab
|
||||
following **Settings > CI/CD** and look for the "Secret Variables" section.
|
||||
As **Key** add the name `SSH_PRIVATE_KEY` and in the **Value** field paste the
|
||||
content of your _private_ key that you created earlier.
|
||||
1. You will first need to create an SSH key pair. For more information, follow
|
||||
the instructions to [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair).
|
||||
**Do not** add a passphrase to the SSH key, or the `before_script` will\
|
||||
prompt for it.
|
||||
|
||||
It is also good practice to check the server's own public key to make sure you
|
||||
are not being targeted by a man-in-the-middle attack. To do this, add another
|
||||
variable named `SSH_SERVER_HOSTKEYS`. To find out the hostkeys of your server, run
|
||||
the `ssh-keyscan YOUR_SERVER` command from a trusted network (ideally, from the
|
||||
server itself), and paste its output into the `SSH_SERVER_HOSTKEYS` variable. If
|
||||
you need to connect to multiple servers, concatenate all the server public keys
|
||||
that you collected into the **Value** of the variable. There must be one key per
|
||||
line.
|
||||
1. Create a new [secret variable](../variables/README.md#secret-variables).
|
||||
As **Key** enter the name `SSH_PRIVATE_KEY` and in the **Value** field paste
|
||||
the content of your _private_ key that you created earlier.
|
||||
|
||||
Next you need to modify your `.gitlab-ci.yml` with a `before_script` action.
|
||||
Add it to the top:
|
||||
1. Modify your `.gitlab-ci.yml` with a `before_script` action. In the following
|
||||
example, a Debian based image is assumed. Edit to your needs:
|
||||
|
||||
```
|
||||
before_script:
|
||||
# Install ssh-agent if not already installed, it is required by Docker.
|
||||
# (change apt-get to yum if you use a CentOS-based image)
|
||||
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
|
||||
```yaml
|
||||
before_script:
|
||||
##
|
||||
## Install ssh-agent if not already installed, it is required by Docker.
|
||||
## (change apt-get to yum if you use an RPM-based image)
|
||||
##
|
||||
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
|
||||
|
||||
# Run ssh-agent (inside the build environment)
|
||||
- eval $(ssh-agent -s)
|
||||
##
|
||||
## Run ssh-agent (inside the build environment)
|
||||
##
|
||||
- eval $(ssh-agent -s)
|
||||
|
||||
# Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
|
||||
- ssh-add <(echo "$SSH_PRIVATE_KEY")
|
||||
##
|
||||
## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
|
||||
## We're using tr to fix line endings which makes ed25519 keys work
|
||||
## without extra base64 encoding.
|
||||
## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
|
||||
##
|
||||
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
|
||||
|
||||
# For Docker builds disable host key checking. Be aware that by adding that
|
||||
# you are suspectible to man-in-the-middle attacks.
|
||||
# WARNING: Use this only with the Docker executor, if you use it with shell
|
||||
# you will overwrite your user's SSH config.
|
||||
- mkdir -p ~/.ssh
|
||||
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
|
||||
# In order to properly check the server's host key, assuming you created the
|
||||
# SSH_SERVER_HOSTKEYS variable previously, uncomment the following two lines
|
||||
# instead.
|
||||
# - mkdir -p ~/.ssh
|
||||
# - '[[ -f /.dockerenv ]] && echo "$SSH_SERVER_HOSTKEYS" > ~/.ssh/known_hosts'
|
||||
```
|
||||
##
|
||||
## Create the SSH directory and give it the right permissions
|
||||
##
|
||||
- mkdir -p ~/.ssh
|
||||
- chmod 700 ~/.ssh
|
||||
|
||||
As a final step, add the _public_ key from the one you created earlier to the
|
||||
services that you want to have an access to from within the build environment.
|
||||
If you are accessing a private GitLab repository you need to add it as a
|
||||
[deploy key](../../ssh/README.md#deploy-keys).
|
||||
##
|
||||
## Optionally, if you will be using any Git commands, set the user name and
|
||||
## and email.
|
||||
##
|
||||
#- git config --global user.email "user@example.com"
|
||||
#- git config --global user.name "User name"
|
||||
```
|
||||
|
||||
NOTE: **Note:**
|
||||
The [`before_script`](../yaml/README.md#before-script) can be set globally
|
||||
or per-job.
|
||||
|
||||
1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys).
|
||||
|
||||
1. As a final step, add the _public_ key from the one you created in the first
|
||||
step to the services that you want to have an access to from within the build
|
||||
environment. If you are accessing a private GitLab repository you need to add
|
||||
it as a [deploy key](../../ssh/README.md#deploy-keys).
|
||||
|
||||
That's it! You can now have access to private servers or repositories in your
|
||||
build environment.
|
||||
|
@ -91,24 +113,93 @@ SSH key.
|
|||
You can generate the SSH key from the machine that GitLab Runner is installed
|
||||
on, and use that key for all projects that are run on this machine.
|
||||
|
||||
First, you need to login to the server that runs your jobs.
|
||||
1. First, you need to login to the server that runs your jobs.
|
||||
|
||||
Then from the terminal login as the `gitlab-runner` user and generate the SSH
|
||||
key pair as described in the [SSH keys documentation](../../ssh/README.md).
|
||||
1. Then from the terminal login as the `gitlab-runner` user:
|
||||
|
||||
As a final step, add the _public_ key from the one you created earlier to the
|
||||
services that you want to have an access to from within the build environment.
|
||||
If you are accessing a private GitLab repository you need to add it as a
|
||||
[deploy key](../../ssh/README.md#deploy-keys).
|
||||
```
|
||||
sudo su - gitlab-runner
|
||||
```
|
||||
|
||||
1. Generate the SSH key pair as described in the instructions to
|
||||
[generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair).
|
||||
**Do not** add a passphrase to the SSH key, or the `before_script` will
|
||||
prompt for it.
|
||||
|
||||
1. As a final step, add the _public_ key from the one you created earlier to the
|
||||
services that you want to have an access to from within the build environment.
|
||||
If you are accessing a private GitLab repository you need to add it as a
|
||||
[deploy key](../../ssh/README.md#deploy-keys).
|
||||
|
||||
Once done, try to login to the remote server in order to accept the fingerprint:
|
||||
|
||||
```bash
|
||||
ssh <address-of-my-server>
|
||||
ssh example.com
|
||||
```
|
||||
|
||||
For accessing repositories on GitLab.com, the `<address-of-my-server>` would be
|
||||
`git@gitlab.com`.
|
||||
For accessing repositories on GitLab.com, you would use `git@gitlab.com`.
|
||||
|
||||
## Verifying the SSH host keys
|
||||
|
||||
It is a good practice to check the private server's own public key to make sure
|
||||
you are not being targeted by a man-in-the-middle attack. In case anything
|
||||
suspicious happens, you will notice it since the job would fail (the SSH
|
||||
connection would fail if the public keys would not match).
|
||||
|
||||
To find out the host keys of your server, run the `ssh-keyscan` command from a
|
||||
trusted network (ideally, from the private server itself):
|
||||
|
||||
```sh
|
||||
## Use the domain name
|
||||
ssh-keyscan example.com
|
||||
|
||||
## Or use an IP
|
||||
ssh-keyscan 1.2.3.4
|
||||
```
|
||||
|
||||
Create a new [secret variable](../variables/README.md#secret-variables) with
|
||||
`SSH_KNOWN_HOSTS` as "Key", and as a "Value" add the output of `ssh-keyscan`.
|
||||
|
||||
NOTE: **Note:**
|
||||
If you need to connect to multiple servers, all the server host keys
|
||||
need to be collected in the **Value** of the variable, one key per line.
|
||||
|
||||
TIP: **Tip:**
|
||||
By using a secret variable instead of `ssh-keyscan` directly inside
|
||||
`.gitlab-ci.yml`, it has the benefit that you don't have to change `.gitlab-ci.yml`
|
||||
if the host domain name changes for some reason. Also, the values are predefined
|
||||
by you, meaning that if the host keys suddenly change, the CI/CD job will fail,
|
||||
and you'll know there's something wrong with the server or the network.
|
||||
|
||||
Now that the `SSH_KNOWN_HOSTS` variable is created, in addition to the
|
||||
[content of `.gitlab-ci.yml`](#ssh-keys-when-using-the-docker-executor)
|
||||
above, here's what more you need to add:
|
||||
|
||||
```yaml
|
||||
before_script:
|
||||
##
|
||||
## Assuming you created the SSH_KNOWN_HOSTS variable, uncomment the
|
||||
## following two lines.
|
||||
##
|
||||
- echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts'
|
||||
- chmod 644 ~/.ssh/known_hosts
|
||||
|
||||
##
|
||||
## Alternatively, use ssh-keyscan to scan the keys of your private server.
|
||||
## Replace example.com with your private server's domain name. Repeat that
|
||||
## command if you have more than one server to connect to.
|
||||
##
|
||||
#- ssh-keyscan example.com >> ~/.ssh/known_hosts
|
||||
#- chmod 644 ~/.ssh/known_hosts
|
||||
|
||||
##
|
||||
## You can optionally disable host key checking. Be aware that by adding that
|
||||
## you are suspectible to man-in-the-middle attacks.
|
||||
## WARNING: Use this only with the Docker executor, if you use it with shell
|
||||
## you will overwrite your user's SSH config.
|
||||
##
|
||||
#- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
|
||||
```
|
||||
|
||||
## Example project
|
||||
|
||||
|
@ -119,6 +210,4 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available
|
|||
Want to hack on it? Simply fork it, commit and push your changes. Within a few
|
||||
moments the changes will be picked by a public runner and the job will begin.
|
||||
|
||||
[ssh-keygen]: http://linux.die.net/man/1/ssh-keygen
|
||||
[ssh-agent]: http://linux.die.net/man/1/ssh-agent
|
||||
[ssh-example-repo]: https://gitlab.com/gitlab-examples/ssh-private-key/
|
||||
|
|
|
@ -213,14 +213,15 @@ An example project service that defines deployment variables is
|
|||
## Debug tracing
|
||||
|
||||
> Introduced in GitLab Runner 1.7.
|
||||
>
|
||||
> **WARNING:** Enabling debug tracing can have severe security implications. The
|
||||
output **will** contain the content of all your secret variables and any other
|
||||
secrets! The output **will** be uploaded to the GitLab server and made visible
|
||||
in job traces!
|
||||
|
||||
CAUTION: **Warning:**
|
||||
Enabling debug tracing can have severe security implications. The
|
||||
output **will** contain the content of all your secret variables and any other
|
||||
secrets! The output **will** be uploaded to the GitLab server and made visible
|
||||
in job traces!
|
||||
|
||||
By default, GitLab Runner hides most of the details of what it is doing when
|
||||
processing a job. This behaviour keeps job traces short, and prevents secrets
|
||||
processing a job. This behavior keeps job traces short, and prevents secrets
|
||||
from being leaked into the trace unless your script writes them to the screen.
|
||||
|
||||
If a job isn't working as expected, this can make the problem difficult to
|
||||
|
|
|
@ -68,7 +68,7 @@ in the pipelines settings page.
|
|||
|
||||
Access to pipelines and job details (including output of logs and artifacts)
|
||||
is checked against your current user access level and the **Public pipelines**
|
||||
project setting.
|
||||
project setting under your project's **Settings > CI/CD > General pipelines settings**.
|
||||
|
||||
If **Public pipelines** is enabled (default):
|
||||
|
||||
|
|
Loading…
Reference in a new issue