gitlab-org--gitlab-foss/doc/user/admin_area/monitoring/health_check.md

stage	group	info	type
none	unassigned	To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments	concepts, howto

Health Check (CORE ONLY)

• Liveness and readiness probes were introduced in GitLab 9.1.
• The health_check endpoint was introduced in GitLab 8.8 and was deprecated in GitLab 9.1.
• Access token has been deprecated in GitLab 9.4 in favor of IP whitelist.

GitLab provides liveness and readiness probes to indicate service health and reachability to required services. These probes report on the status of the database connection, Redis connection, and access to the filesystem. These endpoints can be provided to schedulers like Kubernetes to hold traffic until the system is ready or restart the container as needed.

IP whitelist

To access monitoring resources, the requesting client IP needs to be included in a whitelist. For details, see how to add IPs to a whitelist for the monitoring endpoints.

Using the endpoints locally

With default whitelist settings, the probes can be accessed from localhost using the following URLs:

GET http://localhost/-/health

GET http://localhost/-/readiness

GET http://localhost/-/liveness

Health

Checks whether the application server is running. It does not verify the database or other services are running. This endpoint circumvents Rails Controllers and is implemented as additional middleware BasicHealthCheck very early into the request processing lifecycle.

GET /-/health

Example request:

curl "https://gitlab.example.com/-/health"

Example response:

GitLab OK

Readiness

The readiness probe checks whether the GitLab instance is ready to accept traffic via Rails Controllers. The check by default does validate only instance-checks.

If the all=1 parameter is specified, the check also validates the dependent services (Database, Redis, Gitaly etc.) and gives a status for each.

GET /-/readiness
GET /-/readiness?all=1

Example request:

curl "https://gitlab.example.com/-/readiness"

Example response:

{
   "master_check":[{
      "status":"failed",
      "message": "unexpected Master check result: false"
   }],
   ...
}

On failure, the endpoint returns a 503 HTTP status code.

This check does hit the database and Redis if authenticated via token.

This check is being exempt from Rack Attack.

Liveness

WARNING: In GitLab 12.4 the response body of the Liveness check was changed to match the example below.

Checks whether the application server is running. This probe is used to know if Rails Controllers are not deadlocked due to a multi-threading.

GET /-/liveness

Example request:

curl "https://gitlab.example.com/-/liveness"

Example response:

On success, the endpoint returns a 200 HTTP status code, and a response like below.

{
   "status": "ok"
}

On failure, the endpoint returns a 503 HTTP status code.

This check is being exempt from Rack Attack.

Access token (Deprecated)

NOTE: Access token has been deprecated in GitLab 9.4 in favor of IP whitelist.

An access token needs to be provided while accessing the probe endpoints. The current accepted token can be found under the Admin Area > Monitoring > Health check (admin/health_check) page of your GitLab instance.

The access token can be passed as a URL parameter:

https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN

NOTE: In case the database or Redis service are inaccessible, the probe endpoints response is not guaranteed to be correct. You should switch to IP whitelist from deprecated access token to avoid it.