Merge branch 'docs/prometheus' into 'master'
Update Prometheus documentation Closes #27735 See merge request !9060
This commit is contained in:
commit
0fddece769
|
@ -56,7 +56,7 @@
|
|||
- [Housekeeping](administration/housekeeping.md) Keep your Git repository tidy and fast.
|
||||
- [GitLab Pages configuration](administration/pages/index.md) Configure GitLab Pages.
|
||||
- [GitLab performance monitoring with InfluxDB](administration/monitoring/performance/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics.
|
||||
- [GitLab performance monitoring with Prometheus](administration/monitoring/performance/prometheus.md) Configure GitLab and Prometheus for measuring performance metrics.
|
||||
- [GitLab performance monitoring with Prometheus](administration/monitoring/prometheus/index.md) Configure GitLab and Prometheus for measuring performance metrics.
|
||||
- [Request Profiling](administration/monitoring/performance/request_profiling.md) Get a detailed profile on slow requests.
|
||||
- [Monitoring uptime](user/admin_area/monitoring/health_check.md) Check the server status using the health check endpoint.
|
||||
- [Debugging Tips](administration/troubleshooting/debug.md) Tips to debug problems when things go wrong
|
||||
|
|
|
@ -15,7 +15,7 @@ documents in order to understand and properly configure GitLab Performance Monit
|
|||
>**Note:**
|
||||
Omnibus GitLab 8.16 includes Prometheus as an additional tool to collect
|
||||
metrics. It will eventually replace InfluxDB when their metrics collection is
|
||||
on par. Read more in the [Prometheus documentation](prometheus.md).
|
||||
on par. Read more in the [Prometheus documentation](../prometheus/index.md).
|
||||
|
||||
## Introduction to GitLab Performance Monitoring
|
||||
|
||||
|
|
|
@ -1,102 +1 @@
|
|||
# GitLab Prometheus
|
||||
|
||||
>**Notes:**
|
||||
- Prometheus and the node exporter are bundled in the Omnibus GitLab package
|
||||
since GitLab 8.16. For installations from source you will have to install
|
||||
them yourself. Over subsequent releases additional GitLab metrics will be
|
||||
captured.
|
||||
- Prometheus services are off by default but will be on starting with GitLab 9.0.
|
||||
|
||||
[Prometheus] is a powerful time-series monitoring service, providing a flexible
|
||||
platform for monitoring GitLab and other software products.
|
||||
GitLab provides out of the box monitoring with Prometheus, providing easy
|
||||
access to high quality time-series monitoring of GitLab services.
|
||||
|
||||
## Overview
|
||||
|
||||
Prometheus works by periodically connecting to data sources and collecting their
|
||||
performance metrics. To view and work with the monitoring data, you can either
|
||||
connect directly to Prometheus or utilize a dashboard tool like [Grafana].
|
||||
|
||||
## Configuring Prometheus
|
||||
|
||||
>**Note:**
|
||||
Available since Omnibus GitLab 8.16. For installations from source you'll
|
||||
have to install and configure it yourself.
|
||||
|
||||
To enable Prometheus:
|
||||
|
||||
1. Edit `/etc/gitlab/gitlab.rb`
|
||||
1. Find and uncomment the following line, making sure it's set to `true`:
|
||||
|
||||
```ruby
|
||||
prometheus['enable'] = true
|
||||
```
|
||||
|
||||
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
|
||||
take effect
|
||||
|
||||
By default, Prometheus will run as the `gitlab-prometheus` user and listen on
|
||||
TCP port `9090` under localhost. If the [node exporter](#node-exporter) service
|
||||
has been enabled, it will automatically be set up as a monitoring target for
|
||||
Prometheus.
|
||||
|
||||
## Viewing Performance Metrics
|
||||
|
||||
After you have [enabled Prometheus](#configuring-prometheus), you can visit
|
||||
`<your_domain_name>:9090` for the dashboard that Prometheus offers by default.
|
||||
|
||||
The performance data collected by Prometheus can be viewed directly in the
|
||||
Prometheus console or through a compatible dashboard tool.
|
||||
The Prometheus interface provides a [flexible query language][prom-query] to work
|
||||
with the collected data where you can visualize their output.
|
||||
For a more fully featured dashboard, Grafana can be used and has
|
||||
[official support for Prometheus][prom-grafana].
|
||||
|
||||
## Prometheus exporters
|
||||
|
||||
There are a number of libraries and servers which help in exporting existing
|
||||
metrics from third-party systems as Prometheus metrics. This is useful for cases
|
||||
where it is not feasible to instrument a given system with Prometheus metrics
|
||||
directly (for example, HAProxy or Linux system stats). You can read more in the
|
||||
[Prometheus exporters and integrations documentation][prom-exporters].
|
||||
|
||||
While you can use any exporter you like with your GitLab installation, the
|
||||
following ones documented here are bundled in the Omnibus GitLab packages
|
||||
making it easy to configure and use.
|
||||
|
||||
### Node exporter
|
||||
|
||||
>**Note:**
|
||||
Available since Omnibus GitLab 8.16. For installations from source you'll
|
||||
have to install and configure it yourself.
|
||||
|
||||
The [node exporter] allows you to measure various machine resources such as
|
||||
memory, disk and CPU utilization.
|
||||
|
||||
To enable the node exporter:
|
||||
|
||||
1. [Enable Prometheus](#configuring-prometheus)
|
||||
1. Edit `/etc/gitlab/gitlab.rb`
|
||||
1. Find and uncomment the following line, making sure it's set to `true`:
|
||||
|
||||
```ruby
|
||||
node_exporter['enable'] = true
|
||||
```
|
||||
|
||||
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
|
||||
take effect
|
||||
|
||||
Prometheus it will now automatically begin collecting performance data from
|
||||
the node exporter. You can visit `<your_domain_name>:9100/metrics` for a real
|
||||
time representation of the metrics that are collected. Refresh the page and
|
||||
you will see the data change.
|
||||
|
||||
[grafana]: https://grafana.net
|
||||
[node exporter]: https://github.com/prometheus/node_exporter
|
||||
[prometheus]: https://prometheus.io
|
||||
[prom-query]: https://prometheus.io/docs/querying/basics
|
||||
[prom-grafana]: https://prometheus.io/docs/visualization/grafana/
|
||||
[scrape-config]: https://prometheus.io/docs/operating/configuration/#%3Cscrape_config%3E
|
||||
[prom-exporters]: https://prometheus.io/docs/instrumenting/exporters/
|
||||
[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure
|
||||
This document was moved to [monitoring/prometheus](../prometheus/index.md).
|
||||
|
|
|
@ -0,0 +1,30 @@
|
|||
# GitLab monitor exporter
|
||||
|
||||
>**Note:**
|
||||
Available since [Omnibus GitLab 8.17][1132]. For installations from source
|
||||
you'll have to install and configure it yourself.
|
||||
|
||||
The [GitLab monitor exporter] allows you to measure various GitLab metrics.
|
||||
|
||||
To enable the GitLab monitor exporter:
|
||||
|
||||
1. [Enable Prometheus](index.md#configuring-prometheus)
|
||||
1. Edit `/etc/gitlab/gitlab.rb`
|
||||
1. Add or find and uncomment the following line, making sure it's set to `true`:
|
||||
|
||||
```ruby
|
||||
gitlab_monitor_exporter['enable'] = true
|
||||
```
|
||||
|
||||
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
|
||||
take effect
|
||||
|
||||
Prometheus will now automatically begin collecting performance data from
|
||||
the GitLab monitor exporter exposed under `localhost:9168`.
|
||||
|
||||
[← Back to the main Prometheus page](index.md)
|
||||
|
||||
[1132]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1132
|
||||
[GitLab monitor exporter]: https://gitlab.com/gitlab-org/gitlab-monitor
|
||||
[prometheus]: https://prometheus.io
|
||||
[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure
|
|
@ -0,0 +1,147 @@
|
|||
# GitLab Prometheus
|
||||
|
||||
>**Notes:**
|
||||
- Prometheus and the various exporters listed in this page are bundled in the
|
||||
Omnibus GitLab package. Check each exporter's documentation for the timeline
|
||||
they got added. For installations from source you will have to install
|
||||
them yourself. Over subsequent releases additional GitLab metrics will be
|
||||
captured.
|
||||
- Prometheus services are off by default but will be on starting with GitLab 9.0.
|
||||
- Prometheus and its exporters do not authenticate users, and will be available
|
||||
to anyone who can access them.
|
||||
|
||||
[Prometheus] is a powerful time-series monitoring service, providing a flexible
|
||||
platform for monitoring GitLab and other software products.
|
||||
GitLab provides out of the box monitoring with Prometheus, providing easy
|
||||
access to high quality time-series monitoring of GitLab services.
|
||||
|
||||
## Overview
|
||||
|
||||
Prometheus works by periodically connecting to data sources and collecting their
|
||||
performance metrics via the [various exporters](#prometheus-exporters). To view
|
||||
and work with the monitoring data, you can either
|
||||
[connect directly to Prometheus](#viewing-performance-metrics) or utilize a
|
||||
dashboard tool like [Grafana].
|
||||
|
||||
## Configuring Prometheus
|
||||
|
||||
>**Note:**
|
||||
Available since Omnibus GitLab 8.16. For installations from source you'll
|
||||
have to install and configure it yourself.
|
||||
|
||||
To enable Prometheus:
|
||||
|
||||
1. Edit `/etc/gitlab/gitlab.rb`
|
||||
1. Add or find and uncomment the following line, making sure it's set to `true`:
|
||||
|
||||
```ruby
|
||||
prometheus['enable'] = true
|
||||
```
|
||||
|
||||
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
|
||||
take effect
|
||||
|
||||
By default, Prometheus will run as the `gitlab-prometheus` user and listen on
|
||||
`http://localhost:9090`. If the [node exporter](#node-exporter) service
|
||||
has been enabled, it will automatically be set up as a monitoring target for
|
||||
Prometheus.
|
||||
|
||||
## Changing the port Prometheus listens on
|
||||
|
||||
>**Note:**
|
||||
The following change was added in [GitLab Omnibus 8.17][1261]. Although possible,
|
||||
it's not recommended to change the default address and port Prometheus listens
|
||||
on as this might affect or conflict with other services running on the GitLab
|
||||
server. Proceed at your own risk.
|
||||
|
||||
To change the address/port that Prometheus listens on:
|
||||
|
||||
1. Edit `/etc/gitlab/gitlab.rb`
|
||||
1. Add or find and uncomment the following line:
|
||||
|
||||
```ruby
|
||||
prometheus['listen_address'] = 'localhost:9090'
|
||||
```
|
||||
|
||||
Replace `localhost:9090` with the address/port you want Prometheus to
|
||||
listen on.
|
||||
|
||||
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
|
||||
take effect
|
||||
|
||||
## Viewing performance metrics
|
||||
|
||||
After you have [enabled Prometheus](#configuring-prometheus), you can visit
|
||||
`http://localhost:9090` for the dashboard that Prometheus offers by default.
|
||||
|
||||
>**Note:**
|
||||
If SSL has been enabled on your GitLab instance, you may not be able to access
|
||||
Prometheus on the same browser as GitLab due to [HSTS][hsts]. We plan to
|
||||
[provide access via GitLab][multi-user-prometheus], but in the interim there are
|
||||
some workarounds: using a separate browser for Prometheus, resetting HSTS, or
|
||||
having [Nginx proxy it][nginx-custom-config]. Follow issue [#27069] for more
|
||||
information.
|
||||
|
||||
The performance data collected by Prometheus can be viewed directly in the
|
||||
Prometheus console or through a compatible dashboard tool.
|
||||
The Prometheus interface provides a [flexible query language][prom-query] to work
|
||||
with the collected data where you can visualize their output.
|
||||
For a more fully featured dashboard, Grafana can be used and has
|
||||
[official support for Prometheus][prom-grafana].
|
||||
|
||||
Sample Prometheus queries:
|
||||
|
||||
- **% Memory used:** `(1 - ((node_memory_MemFree + node_memory_Cached) / node_memory_MemTotal)) * 100`
|
||||
- **% CPU load:** `1 - rate(node_cpu{mode="idle"}[5m])`
|
||||
- **Data transmitted:** `irate(node_network_transmit_bytes[5m])`
|
||||
- **Data received:** `irate(node_network_receive_bytes[5m])`
|
||||
|
||||
## Prometheus exporters
|
||||
|
||||
There are a number of libraries and servers which help in exporting existing
|
||||
metrics from third-party systems as Prometheus metrics. This is useful for cases
|
||||
where it is not feasible to instrument a given system with Prometheus metrics
|
||||
directly (for example, HAProxy or Linux system stats). You can read more in the
|
||||
[Prometheus exporters and integrations upstream documentation][prom-exporters].
|
||||
|
||||
While you can use any exporter you like with your GitLab installation, the
|
||||
following ones documented here are bundled in the Omnibus GitLab packages
|
||||
making it easy to configure and use.
|
||||
|
||||
### Node exporter
|
||||
|
||||
The node exporter allows you to measure various machine resources such as
|
||||
memory, disk and CPU utilization.
|
||||
|
||||
[➔ Read more about the node exporter.](node_exporter.md)
|
||||
|
||||
### Redis exporter
|
||||
|
||||
The Redis exporter allows you to measure various Redis metrics.
|
||||
|
||||
[➔ Read more about the Redis exporter.](redis_exporter.md)
|
||||
|
||||
### Postgres exporter
|
||||
|
||||
The Postgres exporter allows you to measure various PostgreSQL metrics.
|
||||
|
||||
[➔ Read more about the Postgres exporter.](postgres_exporter.md)
|
||||
|
||||
### GitLab monitor exporter
|
||||
|
||||
The GitLab monitor exporter allows you to measure various GitLab metrics.
|
||||
|
||||
[➔ Read more about the GitLab monitor exporter.](gitlab_monitor_exporter.md)
|
||||
|
||||
[grafana]: https://grafana.net
|
||||
[hsts]: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
|
||||
[multi-user-prometheus]: https://gitlab.com/gitlab-org/multi-user-prometheus
|
||||
[nginx-custom-config]: https://docs.gitlab.com/omnibus/settings/nginx.html#inserting-custom-nginx-settings-into-the-gitlab-server-block
|
||||
[prometheus]: https://prometheus.io
|
||||
[prom-exporters]: https://prometheus.io/docs/instrumenting/exporters/
|
||||
[prom-query]: https://prometheus.io/docs/querying/basics
|
||||
[prom-grafana]: https://prometheus.io/docs/visualization/grafana/
|
||||
[scrape-config]: https://prometheus.io/docs/operating/configuration/#%3Cscrape_config%3E
|
||||
[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure
|
||||
[#27069]: https://gitlab.com/gitlab-org/gitlab-ce/issues/27069
|
||||
[1261]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1261
|
|
@ -0,0 +1,30 @@
|
|||
# Node exporter
|
||||
|
||||
>**Note:**
|
||||
Available since Omnibus GitLab 8.16. For installations from source you'll
|
||||
have to install and configure it yourself.
|
||||
|
||||
The [node exporter] allows you to measure various machine resources such as
|
||||
memory, disk and CPU utilization.
|
||||
|
||||
To enable the node exporter:
|
||||
|
||||
1. [Enable Prometheus](index.md#configuring-prometheus)
|
||||
1. Edit `/etc/gitlab/gitlab.rb`
|
||||
1. Add or find and uncomment the following line, making sure it's set to `true`:
|
||||
|
||||
```ruby
|
||||
node_exporter['enable'] = true
|
||||
```
|
||||
|
||||
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
|
||||
take effect
|
||||
|
||||
Prometheus will now automatically begin collecting performance data from
|
||||
the node exporter exposed under `localhost:9100`.
|
||||
|
||||
[← Back to the main Prometheus page](index.md)
|
||||
|
||||
[node exporter]: https://github.com/prometheus/node_exporter
|
||||
[prometheus]: https://prometheus.io
|
||||
[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure
|
|
@ -0,0 +1,30 @@
|
|||
# Postgres exporter
|
||||
|
||||
>**Note:**
|
||||
Available since [Omnibus GitLab 8.17][1131]. For installations from source
|
||||
you'll have to install and configure it yourself.
|
||||
|
||||
The [postgres exporter] allows you to measure various PostgreSQL metrics.
|
||||
|
||||
To enable the postgres exporter:
|
||||
|
||||
1. [Enable Prometheus](index.md#configuring-prometheus)
|
||||
1. Edit `/etc/gitlab/gitlab.rb`
|
||||
1. Add or find and uncomment the following line, making sure it's set to `true`:
|
||||
|
||||
```ruby
|
||||
postgres_exporter['enable'] = true
|
||||
```
|
||||
|
||||
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
|
||||
take effect
|
||||
|
||||
Prometheus will now automatically begin collecting performance data from
|
||||
the postgres exporter exposed under `localhost:9187`.
|
||||
|
||||
[← Back to the main Prometheus page](index.md)
|
||||
|
||||
[1131]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1131
|
||||
[postgres exporter]: https://github.com/wrouesnel/postgres_exporter
|
||||
[prometheus]: https://prometheus.io
|
||||
[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure
|
|
@ -0,0 +1,33 @@
|
|||
# Redis exporter
|
||||
|
||||
>**Note:**
|
||||
Available since [Omnibus GitLab 8.17][1118]. For installations from source
|
||||
you'll have to install and configure it yourself.
|
||||
|
||||
The [Redis exporter] allows you to measure various [Redis] metrics. For more
|
||||
information on what's exported [read the upstream documentation][redis-exp].
|
||||
|
||||
To enable the Redis exporter:
|
||||
|
||||
1. [Enable Prometheus](index.md#configuring-prometheus)
|
||||
1. Edit `/etc/gitlab/gitlab.rb`
|
||||
1. Add or find and uncomment the following line, making sure it's set to `true`:
|
||||
|
||||
```ruby
|
||||
redis_exporter['enable'] = true
|
||||
```
|
||||
|
||||
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
|
||||
take effect
|
||||
|
||||
Prometheus will now automatically begin collecting performance data from
|
||||
the Redis exporter exposed under `localhost:9121`.
|
||||
|
||||
[← Back to the main Prometheus page](index.md)
|
||||
|
||||
[1118]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1118
|
||||
[redis]: https://redis.io
|
||||
[redis exporter]: https://github.com/oliver006/redis_exporter
|
||||
[redis-exp]: https://github.com/oliver006/redis_exporter/blob/master/README.md#whats-exported
|
||||
[prometheus]: https://prometheus.io
|
||||
[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure
|
Loading…
Reference in New Issue