Address feedback for AWS Runner autoscale article
This commit is contained in:
parent
1982125081
commit
bca153e538
|
@ -26,7 +26,7 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM
|
|||
|
||||
| Article title | Category | Publishing date |
|
||||
| :------------ | :------: | --------------: |
|
||||
| [Autoscaling GitLab Runners on AWS](runner_autoscale_aws/index.md) | Tutorial | 2017-11-10 |
|
||||
| [Autoscaling GitLab Runners on AWS](runner_autoscale_aws/index.md) | Admin guide | 2017-11-24 |
|
||||
| [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) | Tutorial | 2017-08-31 |
|
||||
| [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md) | Tutorial | 2017-08-15 |
|
||||
| [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017-07-13 |
|
||||
|
|
|
@ -1,7 +1,12 @@
|
|||
---
|
||||
last_updated: 2017-11-10
|
||||
last_updated: 2017-11-24
|
||||
---
|
||||
|
||||
> **[Article Type](../../development/writing_documentation.html#types-of-technical-articles):** Admin guide ||
|
||||
> **Level:** intermediary ||
|
||||
> **Author:** [Achilleas Pipinellis](https://gitlab.com/axil) ||
|
||||
> **Publication date:** 2017/11/24
|
||||
|
||||
# Autoscaling GitLab Runner on AWS
|
||||
|
||||
One of the biggest advantages of GitLab Runner is its ability to automatically
|
||||
|
@ -16,38 +21,43 @@ In this tutorial, we'll explore how to properly configure a GitLab Runner in
|
|||
AWS that will serve as the bastion where it will spawn new Docker machines on
|
||||
demand.
|
||||
|
||||
In addition, we'll make use of [Amazon's EC2 Spot instances](https://aws.amazon.com/ec2/spot/)
|
||||
which will greatly reduce the costs of the Runner instances while still using
|
||||
quite powerful autoscaling machines.
|
||||
In addition, we'll make use of [Amazon's EC2 Spot instances][spot] which will
|
||||
greatly reduce the costs of the Runner instances while still using quite
|
||||
powerful autoscaling machines.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
NOTE: **Note:**
|
||||
A familiarity with Amazon Web Services (AWS) is required as this is where most
|
||||
of the configuration will take place.
|
||||
|
||||
Your GitLab instance is going to need to talk to the Runners over the network,
|
||||
so consider the sensitivity of your projects and communication between nodes
|
||||
when moving forward. This has implications on the AWS security groups you'll
|
||||
need, your DNS configuration, and other networking factors.
|
||||
and that is something you need think about when configuring any AWS security
|
||||
groups or when setting up your DNS configuration.
|
||||
|
||||
For example, you can keep the EC2 resources segmented away from public traffic
|
||||
in a different VPC. Your environment is likely different, so consider what works
|
||||
best for your situation.
|
||||
in a different VPC to better strengthen your network security. Your environment
|
||||
is likely different, so consider what works best for your situation.
|
||||
|
||||
### AWS security groups
|
||||
|
||||
Docker Machine will attempt to use a
|
||||
[default security group](https://docs.docker.com/machine/drivers/aws/#security-group)
|
||||
with rules for port 2376, which is required for communication with the Docker
|
||||
with rules for port `2376`, which is required for communication with the Docker
|
||||
daemon. Instead of relying on Docker, you can create a security group with the
|
||||
rules you need and provide that in the Runner options as we will see below.
|
||||
This way, you can customize it to your liking ahead of time based on your
|
||||
networking environment.
|
||||
rules you need and provide that in the Runner options as we will
|
||||
[see below](#the-runners-machine-section). This way, you can customize it to your
|
||||
liking ahead of time based on your networking environment.
|
||||
|
||||
### AWS credentials
|
||||
|
||||
You'll need an AWS Access Key tied to a user with permission to scale (EC2) and
|
||||
update the cache (via S3). Create a new user with policies for EC2
|
||||
(AmazonEC2FullAccess) and S3 (AmazonS3FullAccess). To be more secure, you can
|
||||
disable console login for that user. Grab the security credentials as we'll use
|
||||
them later during the Runner configuration.
|
||||
You'll need an [AWS Access Key](https://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html)
|
||||
tied to a user with permission to scale (EC2) and update the cache (via S3).
|
||||
Create a new user with [policies](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-policies-for-amazon-ec2.html)
|
||||
for EC2 (AmazonEC2FullAccess) and S3 (AmazonS3FullAccess). To be more secure,
|
||||
you can disable console login for that user. Keep the tab open or copy paste the
|
||||
security credentials in an editor as we'll use them later during the
|
||||
[Runner configuration](#the-runners-machine-section).
|
||||
|
||||
## Prepare the bastion instance
|
||||
|
||||
|
@ -70,11 +80,7 @@ Install the prerequisites:
|
|||
|
||||
Now that the Runner is installed, it's time to register it.
|
||||
|
||||
## Registering GitLab Runner
|
||||
|
||||
TIP: **Tip:**
|
||||
If you want every user in your instance to be able to use the autoscaled Runners,
|
||||
register the Runner as a shared one.
|
||||
## Registering the GitLab Runner
|
||||
|
||||
Before configuring the GitLab Runner, you need to first register it, so that
|
||||
it connects with your GitLab instance:
|
||||
|
@ -83,15 +89,197 @@ it connects with your GitLab instance:
|
|||
1. [Register the Runner](https://docs.gitlab.com/runner/register/index.html#gnu-linux)
|
||||
1. When asked the executor type, enter `docker+machine`
|
||||
|
||||
You can now move on to the most important part, configuring GitLab Runner.
|
||||
You can now move on to the most important part, configuring the GitLab Runner.
|
||||
|
||||
## Configuring GitLab Runner to use the AWS machine driver
|
||||
TIP: **Tip:**
|
||||
If you want every user in your instance to be able to use the autoscaled Runners,
|
||||
register the Runner as a shared one.
|
||||
|
||||
## Configuring the GitLab Runner
|
||||
|
||||
Now that the Runner is registered, you need to edit its configuration file and
|
||||
add the required options for the AWS machine driver.
|
||||
|
||||
Here's a full example of `/etc/gitlab-runner/config.toml`. Open it with your
|
||||
editor and edit as you see fit:
|
||||
Let's first break it down to pieces.
|
||||
|
||||
### The global section
|
||||
|
||||
In the global section, you can define the limit of the jobs that can be run
|
||||
concurrently across all Runners (`concurrent`). This heavily depends on your
|
||||
needs, like how many users your Runners will accommodate, how much time your
|
||||
builds take, etc. You can start with something low like `10`, and increase or
|
||||
decrease its value going forward.
|
||||
|
||||
The `check_interval` option defines how often the Runner should check GitLab
|
||||
for new jobs, in seconds.
|
||||
|
||||
Example:
|
||||
|
||||
```toml
|
||||
concurrent = 10
|
||||
check_interval = 0
|
||||
```
|
||||
|
||||
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section)
|
||||
about all the options you can use.
|
||||
|
||||
### The `runners` section
|
||||
|
||||
From the `[[runners]]` section, the most important part is the `executor` which
|
||||
must be set to `docker+machine`. Most of those settings are taken care of when
|
||||
you register the Runner for the first time.
|
||||
|
||||
`limit` sets the maximum number of machines (running and idle) that this Runner
|
||||
will spawn. For more info check the [relationship between `limit`, `concurrent`
|
||||
and `IdleCount`](https://docs.gitlab.com/runner/configuration/autoscale.html#how-concurrent-limit-and-idlecount-generate-the-upper-limit-of-running-machines).
|
||||
|
||||
Example:
|
||||
|
||||
```toml
|
||||
[[runners]]
|
||||
name = "gitlab-aws-autoscaler"
|
||||
url = "<URL of your GitLab instance>"
|
||||
token = "<Runner's token>"
|
||||
executor = "docker+machine"
|
||||
limit = 20
|
||||
```
|
||||
|
||||
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section)
|
||||
about all the options you can use under `[[runners]]`.
|
||||
|
||||
### The `runners.docker` section
|
||||
|
||||
In the `[runners.docker]` section you can define the default Docker image to
|
||||
be used by the child Runners if it's not defined in [`.gitlab-ci.yml`](../../ci/yaml/README.md).
|
||||
By using `privileged = true`, all Runners will be able to run
|
||||
[Docker in Docker](../../ci/docker/using_docker_build.md#use-docker-in-docker-executor)
|
||||
which is useful if you plan to build your own Docker images via GitLab CI/CD.
|
||||
|
||||
Next, we use `disable_cache = true` to disable the Docker executor's inner
|
||||
cache mechanism since we will use the distributed cache mode as described
|
||||
in the following section.
|
||||
|
||||
Example:
|
||||
|
||||
```toml
|
||||
[runners.docker]
|
||||
image = "alpine"
|
||||
privileged = true
|
||||
disable_cache = true
|
||||
```
|
||||
|
||||
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-docker-section)
|
||||
about all the options you can use under `[runners.docker]`.
|
||||
|
||||
### The `runners.cache` section
|
||||
|
||||
To speed up your jobs, GitLab Runner provides a cache mechanism where selected
|
||||
directories and/or files are saved and shared between subsequent jobs.
|
||||
While not required for this setup, it is recommended to use the distributed cache
|
||||
mechanism that GitLab Runner provides. Since new instances will be created on
|
||||
demand, it is essential to have a common place where the cache is stored.
|
||||
|
||||
In the following example, we use Amazon S3:
|
||||
|
||||
```toml
|
||||
[runners.cache]
|
||||
Type = "s3"
|
||||
ServerAddress = "s3.amazonaws.com"
|
||||
AccessKey = "<your AWS Access Key ID>"
|
||||
SecretKey = "<your AWS Secret Access Key>"
|
||||
BucketName = "<the bucket where your cache should be kept>"
|
||||
BucketLocation = "us-east-1"
|
||||
Shared = true
|
||||
```
|
||||
|
||||
Here's some more info to further explore the cache mechanism:
|
||||
|
||||
- [Reference for `runners.cache`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-section)
|
||||
- [Deploying and using a cache server for GitLab Runner](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching)
|
||||
- [How cache works](../../ci/yaml/README.md#cache)
|
||||
|
||||
### The `runners.machine` section
|
||||
|
||||
This is the most important part of the configuration and it's the one that
|
||||
tells GitLab Runner how and when to spawn new or remove old Docker Machine
|
||||
instances.
|
||||
|
||||
We will focus on the AWS machine options, for the rest of the settings read
|
||||
about the:
|
||||
|
||||
- [Autoscaling algorithm and the parameters it's based on](https://docs.gitlab.com/runner/configuration/autoscale.html#autoscaling-algorithm-and-parameters) - depends on the needs of your organization
|
||||
- [Off peak time configuration](https://docs.gitlab.com/runner/configuration/autoscale.html#off-peak-time-mode-configuration) - useful when there are regular time periods in your organization when no work is done, for example weekends
|
||||
|
||||
Here's an example of the `runners.machine` section:
|
||||
|
||||
```toml
|
||||
[runners.machine]
|
||||
IdleCount = 1
|
||||
IdleTime = 1800
|
||||
MaxBuilds = 10
|
||||
OffPeakPeriods = [
|
||||
"* * 0-9,18-23 * * mon-fri *",
|
||||
"* * * * * sat,sun *"
|
||||
]
|
||||
OffPeakIdleCount = 0
|
||||
OffPeakIdleTime = 1200
|
||||
MachineDriver = "amazonec2"
|
||||
MachineName = "gitlab-docker-machine-%s"
|
||||
MachineOptions = [
|
||||
"amazonec2-access-key=XXXX",
|
||||
"amazonec2-secret-key=XXXX",
|
||||
"amazonec2-region=us-central-1",
|
||||
"amazonec2-vpc-id=vpc-xxxxx",
|
||||
"amazonec2-subnet-id=subnet-xxxxx",
|
||||
"amazonec2-use-private-address=true",
|
||||
"amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true",
|
||||
"amazonec2-security-group=docker-machine-scaler",
|
||||
"amazonec2-instance-type=m4.2xlarge",
|
||||
]
|
||||
```
|
||||
|
||||
The Docker Machine driver is set to `amazonec2` and the machine name has a
|
||||
standard prefix followed by `%s` (required) that is replaced by the ID of the
|
||||
child Runner: `gitlab-docker-machine-%s`.
|
||||
|
||||
Now, depending on your AWS infrastructure, there are many options you can set up
|
||||
under `MachineOptions`. Below you can see the most common ones.
|
||||
|
||||
| Machine option | Description |
|
||||
| -------------- | ----------- |
|
||||
| `amazonec2-access-key=XXXX` | The AWS access key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). |
|
||||
| `amazonec2-secret-key=XXXX` | The AWS secret key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). |
|
||||
| `amazonec2-region=eu-central-1` | The region to use when launching the instance. You can omit this entirely and the default `us-east-1` will be used. |
|
||||
| `amazonec2-vpc-id=vpc-xxxxx` | Your [VPC ID](https://docs.docker.com/machine/drivers/aws/#vpc-id) to launch the instance in. |
|
||||
| `amazonec2-subnet-id=subnet-xxxx` | The AWS VPC subnet ID. |
|
||||
| `amazonec2-use-private-address=true` | Use the private IP address of Docker Machines, but still create a public IP address. Useful to keep the traffic internal and avoid extra costs.|
|
||||
| `amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true` | AWS extra tag key-value pairs, useful to identify the instances on the AWS console. The "Name" tag is set to the machine name by default. We set the "runner-manager-name" to match the Runner name set in `[[runners]]`, so that we can filter all the EC2 instances created by a specific manager setup. Read more about [using tags in AWS](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html). |
|
||||
| `amazonec2-security-group=docker-machine-scaler` | AWS VPC security group name, see [AWS security groups](#aws-security-groups). |
|
||||
| `amazonec2-instance-type=m4.2xlarge` | The instance type that the child Runners will run on. |
|
||||
|
||||
TIP: **Tip:**
|
||||
Under `MachineOptions` you can add anything that the [AWS Docker Machine driver
|
||||
supports](https://docs.docker.com/machine/drivers/aws/#options). You are highly
|
||||
encouraged to read Docker's docs as your infrastructure setup may warrant
|
||||
different options to be applied.
|
||||
|
||||
NOTE: **Note:**
|
||||
The child instances will use by default Ubuntu 16.04 unless you choose a
|
||||
different AMI ID by setting `amazonec2-ami`.
|
||||
|
||||
NOTE: **Note:**
|
||||
If you specify `amazonec2-private-address-only=true` as one of the machine
|
||||
options, your EC2 instance won't get assigned a public IP. This is ok if your
|
||||
VPC is configured correctly with an Internet Gateway (IGW) and routing is fine,
|
||||
but it’s something to consider if you've got a more complex configuration. Read
|
||||
more in [Docker docs about VPC connectivity](https://docs.docker.com/machine/drivers/aws/#vpc-connectivity).
|
||||
|
||||
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-machine-section)
|
||||
about all the options you can use under `[runners.machine]`.
|
||||
|
||||
### Getting it all together
|
||||
|
||||
Here's the full example of `/etc/gitlab-runner/config.toml`:
|
||||
|
||||
```toml
|
||||
concurrent = 10
|
||||
|
@ -130,193 +318,19 @@ check_interval = 0
|
|||
MachineOptions = [
|
||||
"amazonec2-access-key=XXXX",
|
||||
"amazonec2-secret-key=XXXX",
|
||||
"amazonec2-region=us-east-1",
|
||||
"amazonec2-region=us-central-1",
|
||||
"amazonec2-vpc-id=vpc-xxxxx",
|
||||
"amazonec2-subnet-id=subnet-xxxxx",
|
||||
"amazonec2-use-private-address=true",
|
||||
"amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true",
|
||||
"amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true",
|
||||
"amazonec2-security-group=docker-machine-scaler",
|
||||
"amazonec2-instance-type=m4.2xlarge",
|
||||
]
|
||||
```
|
||||
|
||||
Let's break it down to pieces.
|
||||
|
||||
### Global section
|
||||
|
||||
In the global section, you can define the limit of the jobs that can be run
|
||||
concurrently across all Runners (`concurrent`). This heavily depends on your
|
||||
needs, like how many users your Runners will accommodate, how much time your
|
||||
builds take, etc. You can start with something low, and increase its value going
|
||||
forward.
|
||||
|
||||
The `check_interval` setting defines in seconds how often the Runner should
|
||||
check GitLab for new jobs.
|
||||
|
||||
Example:
|
||||
|
||||
```toml
|
||||
concurrent = 10
|
||||
check_interval = 0
|
||||
```
|
||||
|
||||
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section)
|
||||
about all the options you can use.
|
||||
|
||||
### `[[runners]]` section
|
||||
|
||||
From the `[[runners]]` section, the most important part is the `executor` which
|
||||
must be set to `docker+machine`. Most of those settings are taken care of when
|
||||
you register the Runner for the first time.
|
||||
|
||||
`limit` sets the maximum number of machines (running and idle) that this Runner
|
||||
will start. For more info check the [relationship between `limit`, `concurrent`
|
||||
and `IdleCount`](https://docs.gitlab.com/runner/configuration/autoscale.html#how-concurrent-limit-and-idlecount-generate-the-upper-limit-of-running-machines).
|
||||
|
||||
Example:
|
||||
|
||||
```toml
|
||||
[[runners]]
|
||||
name = "gitlab-aws-autoscaler"
|
||||
url = "<URL of your GitLab instance>"
|
||||
token = "<Runner's token>"
|
||||
executor = "docker+machine"
|
||||
limit = 20
|
||||
```
|
||||
|
||||
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section)
|
||||
about all the options you can use under `[[runners]]`.
|
||||
|
||||
### `[runners.docker]` section
|
||||
|
||||
In the `[runners.docker]` section you can define the default Docker image to
|
||||
be used by the child Runners if it's not defined in [`.gitlab-ci.yml`](../../ci/yaml/README.md).
|
||||
Using `privileged = true`, all Runners will be able to run Docker in Docker
|
||||
which is useful if you plan to build your own Docker images via GitLab CI/CD.
|
||||
|
||||
Next, we use `disable_cache = true` to disable the Docker executor's inner
|
||||
cache mechanism since we will use the distributed cache mode as described
|
||||
below.
|
||||
|
||||
Example:
|
||||
|
||||
```toml
|
||||
[runners.docker]
|
||||
image = "alpine"
|
||||
privileged = true
|
||||
disable_cache = true
|
||||
```
|
||||
|
||||
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-docker-section)
|
||||
about all the options you can use under `[runners.docker]`.
|
||||
|
||||
### `[runners.cache]` section
|
||||
|
||||
To speed up your jobs, GitLab Runner provides a cache mechanism where selected
|
||||
directories and/or files are saved and shared between subsequent jobs.
|
||||
While not required for this setup, it is recommended to use the distributed cache
|
||||
mechanism that GitLab Runner provides. Since new instances will be created on
|
||||
demand, it is essential to have a common place where cache is stored.
|
||||
|
||||
In the following example, we use Amazon S3:
|
||||
|
||||
```toml
|
||||
[runners.cache]
|
||||
Type = "s3"
|
||||
ServerAddress = "s3.amazonaws.com"
|
||||
AccessKey = "<your AWS Access Key ID>"
|
||||
SecretKey = "<your AWS Secret Access Key>"
|
||||
BucketName = "<the bucket where your cache should be kept>"
|
||||
BucketLocation = "us-east-1"
|
||||
Shared = true
|
||||
```
|
||||
|
||||
Here's some more info to get you started:
|
||||
|
||||
- [The `[runners.cache]` section reference](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-section)
|
||||
- [Deploying and using a cache server for GitLab Runner](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching)
|
||||
- [How cache works](../../ci/yaml/README.md#cache)
|
||||
|
||||
### `[runners.machine]` section
|
||||
|
||||
This is the most important part of the configuration and it's the one that
|
||||
tells GitLab Runner how and when to spawn new or remove old Docker Machine
|
||||
instances.
|
||||
|
||||
We will focus on the AWS machine options, for the rest of the settings read
|
||||
about the:
|
||||
|
||||
- [autoscaling algorithm and the parameters it's based on](https://docs.gitlab.com/runner/configuration/autoscale.html#autoscaling-algorithm-and-parameters) - depends on the needs of your organization
|
||||
- [off peak time configuration](https://docs.gitlab.com/runner/configuration/autoscale.html#off-peak-time-mode-configuration) - useful when there are regular time periods in your organization when no work is done, for example weekends
|
||||
|
||||
Example:
|
||||
|
||||
```toml
|
||||
[runners.machine]
|
||||
IdleCount = 1
|
||||
IdleTime = 1800
|
||||
MaxBuilds = 10
|
||||
OffPeakPeriods = [
|
||||
"* * 0-9,18-23 * * mon-fri *",
|
||||
"* * * * * sat,sun *"
|
||||
]
|
||||
OffPeakIdleCount = 0
|
||||
OffPeakIdleTime = 1200
|
||||
MachineDriver = "amazonec2"
|
||||
MachineName = "gitlab-docker-machine-%s"
|
||||
MachineOptions = [
|
||||
"amazonec2-access-key=XXXX",
|
||||
"amazonec2-secret-key=XXXX",
|
||||
"amazonec2-region=us-east-1",
|
||||
"amazonec2-vpc-id=vpc-xxxxx",
|
||||
"amazonec2-subnet-id=subnet-xxxxx",
|
||||
"amazonec2-use-private-address=true",
|
||||
"amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true",
|
||||
"amazonec2-security-group=docker-machine-scaler",
|
||||
"amazonec2-instance-type=m4.2xlarge",
|
||||
]
|
||||
```
|
||||
|
||||
The Docker Machine driver is set to `amazonec2` and the machine name has a
|
||||
standard prefix followed by `%s` (required) that is replaced by the ID of the
|
||||
child Runner: `gitlab-docker-machine-%s`.
|
||||
|
||||
Now, depending on your AWS infrastructure, there are many options you can set up
|
||||
under `MachineOptions`. Let's see the most common ones:
|
||||
|
||||
- `amazonec2-access-key=XXXX` - The AWS access key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials).
|
||||
- `amazonec2-secret-key=XXXX` - The AWS secret key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials).
|
||||
- `amazonec2-region=eu-central-1` - The region to use when launching the instance. You can omit this entirely and the default `us-east-1` will be used.
|
||||
- `amazonec2-vpc-id=vpc-xxxxx` - Your VPC ID to launch the instance in, read more in [Docker docs about the VPC ID](https://docs.docker.com/machine/drivers/aws/#vpc-id).
|
||||
- `amazonec2-subnet-id=subnet-xxxx` - AWS VPC subnet ID.
|
||||
- `amazonec2-use-private-address=true` - Use the private IP address for docker-machine, but still create a public IP address. Useful to keep the traffic internal and avoid extra costs.
|
||||
- `amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true` - AWS extra tag key-value pairs, useful to identify the instances on the AWS console. Read more about [using tags in AWS](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html).
|
||||
- `amazonec2-security-group=docker-machine-scaler` - AWS VPC security group name, see [AWS security groups](#aws-security-groups).
|
||||
- `amazonec2-instance-type=m4.2xlarge` - The instance type that the child Runners will run on.
|
||||
|
||||
TIP: **Tip:**
|
||||
Under `MachineOptions` you can add anything that the [AWS Docker Machine driver
|
||||
supports](https://docs.docker.com/machine/drivers/aws/#options). You are highly
|
||||
encouraged to read Docker's docs as your infrastructure setup may warrant
|
||||
different options to be applied.
|
||||
|
||||
NOTE: **Note:**
|
||||
The child instances will use by default Ubuntu 16.04 unless you choose a
|
||||
different AMI ID by setting `amazonec2-ami`.
|
||||
|
||||
NOTE: **Note:**
|
||||
If you specify `amazonec2-private-address-only=true` as one of the machine
|
||||
options, your EC2 instance won't get assigned a public IP. This is fine if your
|
||||
VPC is configured correctly with an Internet Gateway (IGW) and routing is fine,
|
||||
but it’s something to consider if you've got a more exotic configuration. Read
|
||||
more in [Docker docs about VPC connectivity](https://docs.docker.com/machine/drivers/aws/#vpc-connectivity).
|
||||
|
||||
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-machine-section)
|
||||
about all the options you can use under `[runners.machine]`.
|
||||
|
||||
## Cutting down costs with Amazon EC2 Spot instances
|
||||
|
||||
As described by Amazon:
|
||||
As [described by][spot] Amazon:
|
||||
|
||||
>
|
||||
Amazon EC2 Spot instances allow you to bid on spare Amazon EC2 computing capacity.
|
||||
|
@ -325,7 +339,7 @@ pricing, you can significantly reduce the cost of running your applications,
|
|||
grow your application’s compute capacity and throughput for the same budget,
|
||||
and enable new types of cloud computing applications.
|
||||
|
||||
In addition to the [`[runners.machine]`](#runners-machine-section) options
|
||||
In addition to the [`runners.machine`](#the-runners-machine-section) options
|
||||
you picked above, in `/etc/gitlab-runner/config.toml` under the `MachineOptions`
|
||||
section, add the following:
|
||||
|
||||
|
@ -339,8 +353,8 @@ section, add the following:
|
|||
|
||||
With this configuration, Docker Machines are created on Spot instances with a
|
||||
maximum bid price of $0.03 per hour and the duration of the Spot instance is
|
||||
capped at 60 minutes. Be sure to check on the current pricing based on the
|
||||
region you picked.
|
||||
capped at 60 minutes. The `0.03` number mentioned above is just an example, so
|
||||
be sure to check on the current pricing based on the region you picked.
|
||||
|
||||
To learn more about Amazon EC2 Spot instances, visit the following links:
|
||||
|
||||
|
@ -370,17 +384,27 @@ docker-machine ls -q --filter state=Error --format "{{.NAME}}"
|
|||
```
|
||||
|
||||
NOTE: **Note:**
|
||||
In [issue 2771](https://gitlab.com/gitlab-org/gitlab-runner/issues/2771)
|
||||
there is a discussion to make GitLab Runner gracefully handle Spot price changes.
|
||||
There are some issues regarding making GitLab Runner gracefully handle Spot
|
||||
price changes, and there are reports of `docker-machine` attempting to
|
||||
continually remove a Docker Machine. GitLab has provided patches for both cases
|
||||
in the upstream project. For more information, see issues
|
||||
[#2771](https://gitlab.com/gitlab-org/gitlab-runner/issues/2771) and
|
||||
[#2772](https://gitlab.com/gitlab-org/gitlab-runner/issues/2772).
|
||||
|
||||
## Conclusion
|
||||
|
||||
In this guide we learned how to install and configure a GitLab Runner in
|
||||
autoscale mode on AWS.
|
||||
|
||||
Using the autoscale feature of GitLab Runner can save you both time and money.
|
||||
Using the Spot instances that AWS provides can save you even more, but you must
|
||||
be aware of the implications. As long as your bid is high enough, there won't
|
||||
be aware of the implications. As long as your bid is high enough, there shouldn't
|
||||
be an issue.
|
||||
|
||||
You can read the following use cases from which this tutorial was influenced:
|
||||
You can read the following use cases from which this tutorial was (heavily)
|
||||
influenced:
|
||||
|
||||
- [HumanGeo - Scaling GitLab CI](http://blog.thehumangeo.com/gitlab-autoscale-runners.html)
|
||||
- [subtrakt Health - Autoscale GitLab CI Runners and save 90% on EC2 costs](https://substrakthealth.com/news/gitlab-ci-cost-savings/)
|
||||
|
||||
[spot]: https://aws.amazon.com/ec2/spot/
|
||||
|
|
Loading…
Reference in New Issue