2019-06-05 12:42:06 +00:00
---
type: reference
2020-05-12 03:09:31 +00:00
stage: Configure
group: Configure
2020-11-26 06:09:20 +00:00
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
2019-06-05 12:42:06 +00:00
---
2021-10-27 21:11:33 +00:00
# Group-level Kubernetes clusters (certificate-based) (DEPRECATED) **(FREE)**
2018-12-07 04:55:19 +00:00
2021-10-27 21:11:33 +00:00
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/34758) in GitLab 11.6.
> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5.
WARNING:
This feature was [deprecated ](https://gitlab.com/groups/gitlab-org/configure/-/epics/8 ) in GitLab 14.5. To connect clusters to GitLab,
2021-12-09 21:11:06 +00:00
use the [GitLab Agent ](../../clusters/agent/index.md ).
2018-12-07 04:55:19 +00:00
2019-05-20 16:51:52 +00:00
Similar to [project-level ](../../project/clusters/index.md ) and
[instance-level ](../../instance/clusters/index.md ) Kubernetes clusters,
group-level Kubernetes clusters allow you to connect a Kubernetes cluster to
your group, enabling you to use the same cluster across multiple projects.
2018-12-07 04:55:19 +00:00
2021-09-21 15:12:11 +00:00
To view your group-level Kubernetes clusters:
1. On the top bar, select **Menu > Groups** and find your group.
1. On the left sidebar, select **Kubernetes** .
2020-09-23 15:10:14 +00:00
2021-06-08 21:10:05 +00:00
## Cluster management project
2018-12-07 04:55:19 +00:00
2021-06-08 21:10:05 +00:00
Attach a [cluster management project ](../../clusters/management_project.md )
to your cluster to manage shared resources requiring `cluster-admin` privileges for
installation, such as an Ingress controller.
2018-12-07 04:55:19 +00:00
## RBAC compatibility
2020-05-22 03:08:28 +00:00
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/29398) in GitLab 11.4.
2021-11-05 15:11:48 +00:00
> - Project namespace restriction was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51716) in GitLab 11.5.
2018-12-07 04:55:19 +00:00
2020-05-08 18:09:55 +00:00
For each project under a group with a Kubernetes cluster, GitLab creates a restricted
service account with [`edit` privileges ](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles )
in the project namespace.
2018-12-07 04:55:19 +00:00
## Cluster precedence
2020-05-08 18:09:55 +00:00
If the project's cluster is available and not disabled, GitLab uses the
project's cluster before using any cluster belonging to the group containing
the project.
2021-02-17 00:09:19 +00:00
In the case of subgroups, GitLab uses the cluster of the closest ancestor group
2018-12-07 04:55:19 +00:00
to the project, provided the cluster is not disabled.
2020-07-08 12:09:33 +00:00
## Multiple Kubernetes clusters
2018-12-07 04:55:19 +00:00
2021-11-05 15:11:48 +00:00
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35094) in GitLab 13.2.
2020-07-08 12:09:33 +00:00
You can associate more than one Kubernetes cluster to your group, and maintain different clusters
2020-05-08 18:09:55 +00:00
for different environments, such as development, staging, and production.
2018-12-07 04:55:19 +00:00
2020-05-08 18:09:55 +00:00
When adding another cluster,
2020-09-07 15:09:04 +00:00
[set an environment scope ](#environment-scopes ) to help
2020-05-08 18:09:55 +00:00
differentiate the new cluster from your other clusters.
2018-12-07 04:55:19 +00:00
2019-06-06 14:32:57 +00:00
## GitLab-managed clusters
2019-05-03 01:05:53 +00:00
2020-02-06 15:09:11 +00:00
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22011) in GitLab 11.5.
> - Became [optional](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26565) in GitLab 11.11.
2019-05-03 01:05:53 +00:00
2020-05-08 18:09:55 +00:00
You can choose to allow GitLab to manage your cluster for you. If GitLab manages
2020-11-19 00:09:41 +00:00
your cluster, resources for your projects are automatically created. See the
2021-10-09 00:12:30 +00:00
[Access controls ](../../project/clusters/cluster_access.md )
2020-05-08 18:09:55 +00:00
section for details on which resources GitLab creates for you.
2019-05-03 01:05:53 +00:00
2020-11-19 00:09:41 +00:00
For clusters not managed by GitLab, project-specific resources aren't created
2020-05-08 18:09:55 +00:00
automatically. If you're using [Auto DevOps ](../../../topics/autodevops/index.md )
2019-11-05 03:06:09 +00:00
for deployments with a cluster not managed by GitLab, you must ensure:
- The project's deployment service account has permissions to deploy to
2021-06-22 19:11:50 +00:00
[`KUBE_NAMESPACE` ](../../project/clusters/deploy_to_cluster.md#deployment-variables ).
2019-11-05 03:06:09 +00:00
- `KUBECONFIG` correctly reflects any changes to `KUBE_NAMESPACE`
2020-05-22 03:08:28 +00:00
(this is [not automatic ](https://gitlab.com/gitlab-org/gitlab/-/issues/31519 )). Editing
2019-11-05 03:06:09 +00:00
`KUBE_NAMESPACE` directly is discouraged.
2019-05-03 01:05:53 +00:00
2019-11-26 09:08:36 +00:00
### Clearing the cluster cache
2020-05-22 03:08:28 +00:00
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31759) in GitLab 12.6.
2019-11-26 09:08:36 +00:00
If you choose to allow GitLab to manage your cluster for you, GitLab stores a cached
version of the namespaces and service accounts it creates for your projects. If you
modify these resources in your cluster manually, this cache can fall out of sync with
your cluster, which can cause deployment jobs to fail.
To clear the cache:
2021-09-21 15:12:11 +00:00
1. On the top bar, select **Menu > Groups** and find your group.
1. On the left sidebar, select **Kubernetes** .
1. Select your cluster.
1. Expand **Advanced settings** .
1. Select **Clear cluster cache** .
2019-11-26 09:08:36 +00:00
2019-01-28 19:42:17 +00:00
## Base domain
2020-02-06 15:09:11 +00:00
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24580) in GitLab 11.8.
2019-01-28 19:42:17 +00:00
Domains at the cluster level permit support for multiple domains
2020-07-08 12:09:33 +00:00
per [multiple Kubernetes clusters ](#multiple-kubernetes-clusters ) When specifying a domain,
2020-11-19 00:09:41 +00:00
this is automatically set as an environment variable (`KUBE_INGRESS_BASE_DOMAIN`) during
2019-01-28 19:42:17 +00:00
the [Auto DevOps ](../../../topics/autodevops/index.md ) stages.
2021-06-22 19:11:50 +00:00
The domain should have a wildcard DNS configured to the Ingress IP address. [More details ](../../project/clusters/gitlab_managed_clusters.md#base-domain ).
2018-12-07 04:55:19 +00:00
2019-07-08 08:50:38 +00:00
## Environment scopes **(PREMIUM)**
2018-12-07 04:55:19 +00:00
2019-03-11 01:47:01 +00:00
When adding more than one Kubernetes cluster to your project, you need to differentiate
them with an environment scope. The environment scope associates clusters with
2020-05-15 18:07:52 +00:00
[environments ](../../../ci/environments/index.md ) similar to how the
2021-06-28 15:08:03 +00:00
[environment-specific CI/CD variables ](../../../ci/variables/index.md#limit-the-environment-scope-of-a-cicd-variable )
2018-12-07 04:55:19 +00:00
work.
While evaluating which environment matches the environment scope of a
2020-05-08 18:09:55 +00:00
cluster, [cluster precedence ](#cluster-precedence ) takes
effect. The cluster at the project level takes precedence, followed
2018-12-07 04:55:19 +00:00
by the closest ancestor group, followed by that groups' parent and so
on.
2020-05-08 18:09:55 +00:00
For example, if your project has the following Kubernetes clusters:
2018-12-07 04:55:19 +00:00
| Cluster | Environment scope | Where |
| ---------- | ------------------- | ----------|
| Project | `*` | Project |
| Staging | `staging/*` | Project |
| Production | `production/*` | Project |
| Test | `test` | Group |
| Development| `*` | Group |
2021-06-28 21:10:13 +00:00
And the following environments are set in [`.gitlab-ci.yml` ](../../../ci/yaml/index.md ):
2018-12-07 04:55:19 +00:00
```yaml
stages:
2020-07-03 09:08:53 +00:00
- test
- deploy
2018-12-07 04:55:19 +00:00
test:
stage: test
script: sh test
deploy to staging:
stage: deploy
script: make deploy
environment:
name: staging/$CI_COMMIT_REF_NAME
url: https://staging.example.com/
deploy to production:
stage: deploy
script: make deploy
environment:
name: production/$CI_COMMIT_REF_NAME
url: https://example.com/
```
2020-05-08 18:09:55 +00:00
The result is:
2018-12-07 04:55:19 +00:00
2020-05-08 18:09:55 +00:00
- The Project cluster is used for the `test` job.
- The Staging cluster is used for the `deploy to staging` job.
- The Production cluster is used for the `deploy to production` job.
2018-12-18 03:13:51 +00:00
2019-09-18 14:02:45 +00:00
## Cluster environments **(PREMIUM)**
2020-05-15 18:07:52 +00:00
For a consolidated view of which CI [environments ](../../../ci/environments/index.md )
2019-10-15 06:07:25 +00:00
are deployed to the Kubernetes cluster, see the documentation for
[cluster environments ](../../clusters/environments.md ).
2019-09-18 14:02:45 +00:00
2020-09-14 03:09:21 +00:00
## Security of runners
2019-07-24 12:25:43 +00:00
2020-09-14 03:09:21 +00:00
For important information about securely configuring runners, see
2021-06-18 15:10:16 +00:00
[Security of runners ](../../project/clusters/cluster_access.md#security-of-runners )
2019-07-24 12:25:43 +00:00
documentation for project-level clusters.
2019-12-20 09:24:38 +00:00
## More information
For information on integrating GitLab and Kubernetes, see
2021-10-09 00:12:30 +00:00
[Kubernetes clusters ](../../infrastructure/clusters/index.md ).
2019-12-20 09:24:38 +00:00
2019-06-06 14:32:57 +00:00
<!-- ## Troubleshooting
Include any troubleshooting steps that you can foresee. If you know beforehand what issues
one might have when setting this up, or when something is changed, or on upgrading, it's
important to describe those, too. Think of things that may go wrong and include them here.
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
Each scenario can be a third-level heading, e.g. `### Getting error message X` .
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->