gitlab-org--gitlab-foss/doc/install/kubernetes/gitlab_chart.md

GitLab Helm Chart

These Helm charts are in beta. GitLab is working on a cloud-native set of Charts which will replace these.

Officially supported cloud providers are Google Container Service and Azure Container Service.

The gitlab Helm chart deploys GitLab into your Kubernetes cluster.

This chart includes the following:

• Deployment using the gitlab-ce or gitlab-ee container image
• ConfigMap containing the gitlab.rb contents that configure Omnibus GitLab
• Persistent Volume Claims for Data, Config, Logs, and Registry Storage
• A Kubernetes service
• Optional Redis deployment using the Redis Chart (defaults to enabled)
• Optional PostgreSQL deployment using the PostgreSQL Chart (defaults to enabled)
• Optional Ingress (defaults to disabled)

Prerequisites

• At least 3 GB of RAM available on your cluster, in chunks of 1 GB. 41GB of storage and 2 CPU are also required.
• Kubernetes 1.4+ with Beta APIs enabled
• Persistent Volume provisioner support in the underlying infrastructure
• The ability to point a DNS entry or URL at your GitLab install
• The kubectl CLI installed locally and authenticated for the cluster
• The Helm Client installed locally
• The Helm Server (Tiller) already installed and running in the cluster, by running helm init
• The GitLab Helm Repo added to your Helm Client

Configuring GitLab

Create a values.yaml file for your GitLab configuration. See the Helm docs for information on how your values file will override the defaults.

The default configuration can always be found in the values.yaml, in the chart repository.

Required configuration

In order for GitLab to function, your config file must specify the following:

• An externalUrl that GitLab will be reachable at.

Choosing GitLab Edition

The Helm chart defaults to installing GitLab CE. This can be controlled by setting the edition variable in your values.

Setting edition to GitLab Enterprise Edition (EE) in your values.yaml

edition: EE

externalUrl: 'http://gitlab.example.com'

Choosing a different GitLab release version

The version of GitLab installed is based on the edition setting (see section above), and the value of the corresponding helm setting: ceImage or eeImage.

## GitLab Edition
## ref: https://about.gitlab.com/products/
## - CE - Community Edition
## - EE - Enterprise Edition - (requires license issued by GitLab Inc)
##
edition: CE

## GitLab CE image
## ref: https://hub.docker.com/r/gitlab/gitlab-ce/tags/
##
ceImage: gitlab/gitlab-ce:9.1.2-ce.0

## GitLab EE image
## ref: https://hub.docker.com/r/gitlab/gitlab-ee/tags/
##
eeImage: gitlab/gitlab-ee:9.1.2-ee.0

The different images can be found in the gitlab-ce and gitlab-ee repositories on Docker Hub

Note: There is no guarantee that other release versions of GitLab, other than what are used by default in the chart, will be supported by a chart install.

Custom Omnibus GitLab configuration

In addition to the configuration options provided for GitLab in the Helm Chart, you can also pass any custom configuration that is valid for the Omnibus GitLab Configuration.

The setting to pass these values in is omnibusConfigRuby. It accepts any valid Ruby code that could used in the Omnibus /etc/gitlab/gitlab.rb file. In Kubernetes, the contents will be stored in a ConfigMap.

Example setting:

omnibusConfigRuby: |
  unicorn['worker_processes'] = 2;
  gitlab_rails['trusted_proxies'] = ["10.0.0.0/8","172.16.0.0/12","192.168.0.0/16"];  

Persistent storage

By default, persistent storage is enabled for GitLab and the charts it depends on (Redis and PostgreSQL).

Components can have their claim size set from your values.yaml, and each component allows you to optionally configure the storageClass variable so you can take advantage of faster drives on your cloud provider.

Basic configuration:

## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
## ref: https://docs.gitlab.com/ce/install/requirements.html#storage
##
persistence:
  ## This volume persists generated configuration files, keys, and certs.
  ##
  gitlabEtc:
    enabled: true
    size: 1Gi
    ## If defined, volume.beta.kubernetes.io/storage-class: <storageClass>
    ## Default: volume.alpha.kubernetes.io/storage-class: default
    ##
    # storageClass:
    accessMode: ReadWriteOnce
  ## This volume is used to store git data and other project files.
  ## ref: https://docs.gitlab.com/omnibus/settings/configuration.html#storing-git-data-in-an-alternative-directory
  ##
  gitlabData:
    enabled: true
    size: 10Gi
    ## If defined, volume.beta.kubernetes.io/storage-class: <storageClass>
    ## Default: volume.alpha.kubernetes.io/storage-class: default
    ##
    # storageClass:
    accessMode: ReadWriteOnce
  gitlabRegistry:
    enabled: true
    size: 10Gi
    ## If defined, volume.beta.kubernetes.io/storage-class: <storageClass>
    ## Default: volume.alpha.kubernetes.io/storage-class: default
    ##
    # storageClass:

  postgresql:
    persistence:
      # storageClass:
      size: 10Gi
  ## Configuration values for the Redis dependency.
  ## ref: https://github.com/kubernetes/charts/blob/master/stable/redis/README.md
  ##
  redis:
    persistence:
      # storageClass:
      size: 10Gi

Note: You can make use of faster SSD drives by adding a StorageClass to your cluster and using the storageClass setting in the above config to the name of your new storage class.

Routing

By default, the GitLab chart uses a service type of LoadBalancer which will result in the GitLab service being exposed externally using your cloud provider's load balancer.

This field is configurable in your values.yml by setting the top-level serviceType field. See the Service documentation for more information on the possible values.

Ingress routing

Optionally, you can enable the Chart's ingress for use by an ingress controller deployed in your cluster.

To enable the ingress, edit its section in your values.yaml:

ingress:
  ## If true, gitlab Ingress will be created
  ##
  enabled: true

  ## gitlab Ingress hostnames
  ## Must be provided if Ingress is enabled
  ##
  hosts:
    - gitlab.example.com

  ## gitlab Ingress annotations
  ##
  annotations:
    kubernetes.io/ingress.class: nginx

You must also provide the list of hosts that the ingress will use. In order for you ingress controller to work with the GitLab Ingress, you will need to specify its class in an annotation.

Note: The Ingress alone doesn't expose GitLab externally. You need to have a Ingress controller setup to do that. Setting up an Ingress controller can be done by installing the nginx-ingress helm chart. But be sure to read the documentation. Note: If you would like to use the Registry, you will also need to ensure your Ingress supports a sufficiently large request size.

Preserving Source IPs

If you are using the LoadBalancer serviceType you may run into issues where user IP addresses in the GitLab logs, and used in abuse throttling are not accurate. This is due to how Kubernetes uses source NATing on cluster nodes without endpoints.

See the Kubernetes documentation for more information.

To fix this you can add the following service annotation to your values.yaml

## For minikube, set this to NodePort, elsewhere use LoadBalancer
## ref: http://kubernetes.io/docs/user-guide/services/#publishing-services---service-types
##
serviceType: LoadBalancer

## Optional annotations for gitlab service.
serviceAnnotations:
  service.beta.kubernetes.io/external-traffic: "OnlyLocal"

Note: If you are using the ingress routing, you will likely also need to specify the annotation on the service for the ingress controller. For nginx-ingress you can check the configuration documentation on how to add the annotation to the controller.service.annotations array.

Note: When using the nginx-ingress controller on Google Container Engine (GKE), and using the external-traffic annotation, you will need to additionally set the controller.kind to be DaemonSet. Otherwise only pods running on the same node as the nginx controller will be able to reach GitLab. This may result in pods within your cluster not being able to reach GitLab. See the Kubernetes documentation and nginx-ingress configuration documentation for more information.

External database

You can configure the GitLab Helm chart to connect to an external PostgreSQL database.

Note: This is currently our recommended approach for a Production setup.

To use an external database, in your values.yaml, disable the included PostgreSQL dependency, then configure access to your database:

dbHost: "<reachable postgres hostname>"
dbPassword: "<password for the user with access to the db>"
dbUsername: "<user with read/write access to the database>"
dbDatabase: "<database name on postgres to connect to for GitLab>"

postgresql:
  # Sets whether the PostgreSQL helm chart is used as a dependency
  enabled: false

Be sure to check the GitLab documentation on how to configure the external database

You can also configure the chart to use an external Redis server, but this is not required for basic production use:

dbHost: "<reachable redis hostname>"
dbPassword: "<password>"

redis:
  # Sets whether the Redis helm chart is used as a dependency
  enabled: false

Sending email

By default, the GitLab container will not be able to send email from your cluster. In order to send email, you should configure SMTP settings in the omnibusConfigRuby section, as per the GitLab Omnibus documentation.

Note: Some cloud providers restrict emails being sent out on SMTP, so you will have to use a SMTP service that is supported by your provider. See this Google Cloud Platform page as and example.

Here is an example configuration for Mailgun SMTP support:

omnibusConfigRuby: |
  # This is example config of what you may already have in your omnibusConfigRuby object
  unicorn['worker_processes'] = 2;
  gitlab_rails['trusted_proxies'] = ["10.0.0.0/8","172.16.0.0/12","192.168.0.0/16"];

  # SMTP settings
  gitlab_rails['smtp_enable'] = true
  gitlab_rails['smtp_address'] = "smtp.mailgun.org"
  gitlab_rails['smtp_port'] = 2525 # High port needed for Google Cloud
  gitlab_rails['smtp_authentication'] = "plain"
  gitlab_rails['smtp_enable_starttls_auto'] = false
  gitlab_rails['smtp_user_name'] = "postmaster@mg.your-mail-domain"
  gitlab_rails['smtp_password'] = "you-password"
  gitlab_rails['smtp_domain'] = "mg.your-mail-domain"  

HTTPS configuration

To setup HTTPS access to your GitLab server, first you need to configure the chart to use the ingress.

GitLab's config should be updated to support proxied SSL.

In addition to having a Ingress Controller deployed and the basic ingress settings configured, you will also need to specify in the ingress settings which hosts to use HTTPS for.

Make sure externalUrl now includes https:// instead of http:// in its value, and update the omnibusConfigRuby section:

externalUrl: 'https://gitlab.example.com'

omnibusConfigRuby: |
  # This is example config of what you may already have in your omnibusConfigRuby object
  unicorn['worker_processes'] = 2;
  gitlab_rails['trusted_proxies'] = ["10.0.0.0/8","172.16.0.0/12","192.168.0.0/16"];

  # These are the settings needed to support proxied SSL
  nginx['listen_port'] = 80
  nginx['listen_https'] = false
  nginx['proxy_set_headers'] = {
    "X-Forwarded-Proto" => "https",
    "X-Forwarded-Ssl" => "on"
  }  

ingress:
  enabled: true
  annotations:
   kubernetes.io/ingress.class: nginx
   # kubernetes.io/tls-acme: 'true' Annotation used for letsencrypt support

  hosts:
    - gitlab.example.com

    ## gitlab Ingress TLS configuration
    ## Secrets must be created in the namespace, and is not done for you in this chart
    ##
    tls:
      - secretName: gitlab-tls
        hosts:
          - gitlab.example.com

You will need to create the named secret in your cluster, specifying the private and public certificate pair using the format outlined in the ingress documentation.

Alternatively, you can use the kubernetes.io/tls-acme annotation, and install the kube-lego chart to your cluster to have Let's Encrypt issue your certificate. See the kube-lego documentation for more information.

Enabling the GitLab Container Registry

The GitLab Registry is disabled by default but can be enabled by providing an external URL for it in the configuration. In order for the Registry to be easily used by GitLab CI and your Kubernetes cluster, you will need to set it up with a TLS certificate, so these examples will include the ingress settings for that as well. See the HTTPS Configuration section for more explanation on some of these settings.

Example config:

externalUrl: 'https://gitlab.example.com'

omnibusConfigRuby: |
  # This is example config of what you may already have in your omnibusConfigRuby object
  unicorn['worker_processes'] = 2;
  gitlab_rails['trusted_proxies'] = ["10.0.0.0/8","172.16.0.0/12","192.168.0.0/16"];

  registry_external_url 'https://registry.example.com';

  # These are the settings needed to support proxied SSL
  nginx['listen_port'] = 80
  nginx['listen_https'] = false
  nginx['proxy_set_headers'] = {
    "X-Forwarded-Proto" => "https",
    "X-Forwarded-Ssl" => "on"
  }
  registry_nginx['listen_port'] = 80
  registry_nginx['listen_https'] = false
  registry_nginx['proxy_set_headers'] = {
    "X-Forwarded-Proto" => "https",
    "X-Forwarded-Ssl" => "on"
  }  

ingress:
  enabled: true
  annotations:
   kubernetes.io/ingress.class: nginx
   # kubernetes.io/tls-acme: 'true' Annotation used for letsencrypt support

  hosts:
    - gitlab.example.com
    - registry.example.com

    ## gitlab Ingress TLS configuration
    ## Secrets must be created in the namespace, and is not done for you in this chart
    ##
    tls:
      - secretName: gitlab-tls
        hosts:
          - gitlab.example.com
          - registry.example.com

Installing GitLab using the Helm Chart

You may see a temporary error message SchedulerPredicates failed due to PersistentVolumeClaim is not bound while storage provisions. Once the storage provisions, the pods will automatically restart. This may take a couple minutes depending on your cloud provider. If the error persists, please review the prerequisites to ensure you have enough RAM, CPU, and storage.

Ensure the GitLab repo has been added and re-initialize Helm:

helm repo add gitlab https://charts.gitlab.io
helm init

Once you have configured GitLab in your values.yml file, run the following:

helm install --namespace <NAMESPACE> --name gitlab -f <CONFIG_VALUES_FILE> gitlab/gitlab

where:

• <NAMESPACE> is the Kubernetes namespace where you want to install GitLab.
• <CONFIG_VALUES_FILE> is the path to values file containing your custom configuration. See the Configuration section to create it.

Updating GitLab using the Helm Chart

Once your GitLab Chart is installed, configuration changes and chart updates should we done using helm upgrade

helm upgrade --namespace <NAMESPACE> -f <CONFIG_VALUES_FILE> <RELEASE-NAME> gitlab/gitlab

where:

• <NAMESPACE> is the Kubernetes namespace where GitLab is installed.
• <CONFIG_VALUES_FILE> is the path to values file containing your custom [configuration] (#configuration).
• <RELEASE-NAME> is the name you gave the chart when installing it. In the Install section we called it gitlab.

Uninstalling GitLab using the Helm Chart

To uninstall the GitLab Chart, run the following:

helm delete --namespace <NAMESPACE> <RELEASE-NAME>

where:

• <NAMESPACE> is the Kubernetes namespace where GitLab is installed.
• <RELEASE-NAME> is the name you gave the chart when installing it. In the Install section we called it gitlab.