diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 22afcb9199d..183808641c0 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -1,26 +1,29 @@ -# Using Docker Build +# Building Docker images with GitLab CI/CD -GitLab CI allows you to use Docker Engine to build and test docker-based projects. +GitLab CI/CD allows you to use Docker Engine to build and test docker-based projects. -**This also allows to you to use `docker-compose` and other docker-enabled tools.** +TIP: **Tip:** +This also allows to you to use `docker-compose` and other docker-enabled tools. One of the new trends in Continuous Integration/Deployment is to: -1. create an application image, -1. run tests against the created image, -1. push image to a remote registry, and -1. deploy to a server from the pushed image. +1. Create an application image +1. Run tests against the created image +1. Push image to a remote registry +1. Deploy to a server from the pushed image -It's also useful when your application already has the `Dockerfile` that can be used to create and test an image: +It's also useful when your application already has the `Dockerfile` that can be +used to create and test an image: ```bash -$ docker build -t my-image dockerfiles/ -$ docker run my-docker-image /script/to/run/tests -$ docker tag my-image my-registry:5000/my-image -$ docker push my-registry:5000/my-image +docker build -t my-image dockerfiles/ +docker run my-docker-image /script/to/run/tests +docker tag my-image my-registry:5000/my-image +docker push my-registry:5000/my-image ``` -This requires special configuration of GitLab Runner to enable `docker` support during jobs. +This requires special configuration of GitLab Runner to enable `docker` support +during jobs. ## Runner Configuration @@ -74,8 +77,8 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user. 5. You can now use `docker` command and install `docker-compose` if needed. -> **Note:** -* By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions. +NOTE: **Note:** +By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions. For more information please read [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). ### Use docker-in-docker executor @@ -259,8 +262,66 @@ aware of the following implications: docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests ``` +## Making docker-in-docker builds faster with Docker layer caching + +When using docker-in-docker, Docker will download all layers of your image every +time you create a build. Recent versions of Docker (Docker 1.13 and above) can +use a pre-existing image as a cache during the `docker build` step, considerably +speeding up the build process. + +### How Docker caching works + +When running `docker build`, each command in `Dockerfile` results in a layer. +These layers are kept around as a cache and can be reused if there haven't been +any changes. Change in one layer causes all subsequent layers to be recreated. + +You can specify a tagged image to be used as a cache source for the `docker build` +command by using the `--cache-from` argument. Multiple images can be specified +as a cache source by using multiple `--cache-from` arguments. Keep in mind that +any image that's used with the `--cache-from` argument must first be pulled +(using `docker pull`) before it can be used as a cache source. + +### Using Docker caching + +Here's a simple `.gitlab-ci.yml` file showing how Docker caching can be utilized: + +```yaml +image: docker:latest + +services: + - docker:dind + +variables: + CONTAINER_IMAGE: registry.gitlab.com/$CI_PROJECT_PATH + DOCKER_DRIVER: overlay2 + +before_script: + - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN registry.gitlab.com + +build: + stage: build + script: + - docker pull $CONTAINER_IMAGE:latest || true + - docker build --cache-from $CONTAINER_IMAGE:latest --tag $CONTAINER_IMAGE:$CI_BUILD_REF --tag $CONTAINER_IMAGE:latest . + - docker push $CONTAINER_IMAGE:$CI_BUILD_REF + - docker push $CONTAINER_IMAGE:latest +``` + +The steps in the `script` section for the `build` stage can be summed up to: + +1. The first command tries to pull the image from the registry so that it can be + used as a cache for the `docker build` command. +1. The second command builds a Docker image using the pulled image as a + cache (notice the `--cache-from $CONTAINER_IMAGE:latest` argument) if + available, and tags it. +1. The last two commands push the tagged Docker images to the container registry + so that they may also be used as cache for subsequent builds. + ## Using the OverlayFS driver +NOTE: **Note:** +The shared Runners on GitLab.com use the `overlay2` driver by default. + By default, when using `docker:dind`, Docker uses the `vfs` storage driver which copies the filesystem on every run. This is a very disk-intensive operation which can be avoided if a different driver is used, for example `overlay2`.