From 9f2a6ca03c9b7bc5e2984bc6a2bf2bf731761784 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kamil=20Trzci=C5=84ski?= Date: Tue, 4 Sep 2018 15:01:34 +0200 Subject: [PATCH] Add development documentation --- doc/development/README.md | 1 + doc/development/prometheus_metrics.md | 46 +++++++++++++++++++ .../prometheus_library/metrics.md | 45 ------------------ 3 files changed, 47 insertions(+), 45 deletions(-) create mode 100644 doc/development/prometheus_metrics.md diff --git a/doc/development/README.md b/doc/development/README.md index 20f8fa1d368..e786d6594c7 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -47,6 +47,7 @@ description: 'Learn how to contribute to GitLab.' - [How to dump production data to staging](db_dump.md) - [Working with the GitHub importer](github_importer.md) - [Working with Merge Request diffs](diffs.md) +- [Prometheus metrics](prometheus_metrics.md) ## Performance guides diff --git a/doc/development/prometheus_metrics.md b/doc/development/prometheus_metrics.md new file mode 100644 index 00000000000..488d344b8cc --- /dev/null +++ b/doc/development/prometheus_metrics.md @@ -0,0 +1,46 @@ +# Working with Prometheus Metrics + +## Adding to the library + +We strive to support the 2-4 most important metrics for each common system service that supports Prometheus. If you are looking for support for a particular exporter which has not yet been added to the library, additions can be made [to the `common_metrics.yml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/prometheus/common_metrics.yml) file. + +### Query identifier + +The requirement for adding a new metrics is to make each query to have an unique identifier. +Identifier is used to update the metric later when changed. + +```yaml +- group: Response metrics (NGINX Ingress) + metrics: + - title: "Throughput" + y_label: "Requests / Sec" + queries: + - id: response_metrics_nginx_ingress_throughput_status_code + query_range: 'sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) by (status_code)' + unit: req / sec + label: Status Code +``` + +### Update existing metrics + +After you add or change existing _common_ metric you have to create a new database migration that will query and update all existing metrics. + +**Note: If a query metric (which is identified by `id:`) is removed it will not be removed from database by default.** +**You might want to add additional database migration that makes a decision what to do with removed one.** +**For example: you might be interested in migrating all dependent data to a different metric.** + +```ruby +class ImportCommonMetrics < ActiveRecord::Migration + require_relative '../importers/common_metrics_importer.rb' + + DOWNTIME = false + + def up + Importers::CommonMetricsImporter.new.execute + end + + def down + # no-op + end +end +``` diff --git a/doc/user/project/integrations/prometheus_library/metrics.md b/doc/user/project/integrations/prometheus_library/metrics.md index 1e0f9d4e249..ec16902fcc8 100644 --- a/doc/user/project/integrations/prometheus_library/metrics.md +++ b/doc/user/project/integrations/prometheus_library/metrics.md @@ -17,48 +17,3 @@ GitLab retrieves performance data from the configured Prometheus server, and att In order to isolate and only display relevant metrics for a given environment, GitLab needs a method to detect which labels are associated. To do that, GitLab uses the defined queries and fills in the environment specific variables. Typically this involves looking for the [$CI_ENVIRONMENT_SLUG](../../../../ci/variables/README.md#predefined-variables-environment-variables), but may also include other information such as the project's Kubernetes namespace. Each search query is defined in the [exporter specific documentation](#prometheus-metrics-library). - -## Adding to the library - -We strive to support the 2-4 most important metrics for each common system service that supports Prometheus. If you are looking for support for a particular exporter which has not yet been added to the library, additions can be made [to the `common_metrics.yml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/prometheus/common_metrics.yml) file. - -### Query identifier - -The requirement for adding metrics is to have each query to have unique identifier. -Identifier is used to update the metric later when changed. - -```yaml -- group: Response metrics (NGINX Ingress) - metrics: - - title: "Throughput" - y_label: "Requests / Sec" - queries: - - id: response_metrics_nginx_ingress_throughput_status_code - query_range: 'sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) by (status_code)' - unit: req / sec - label: Status Code -``` - -### Update existing metrics - -After you add or change existing _common_ metric you have to create a new database migration that will query and update all existing metrics. - -**Note: If a query metric (which is identified by `id:`) is removed it will not be removed from database by default.** -**You might want to add additional database migration that makes a decision what to do with removed one.** -**For example: you might be interested in migrating all dependent data to a different metric.** - -```ruby -class ImportCommonMetrics < ActiveRecord::Migration - require_relative '../importers/common_metrics_importer.rb' - - DOWNTIME = false - - def up - Importers::CommonMetricsImporter.new.execute - end - - def down - # no-op - end -end -```