Add latest changes from gitlab-org/gitlab@master
|
@ -213,7 +213,9 @@ In this example:
|
|||
- Each repository is stored on one of three Gitaly storages: `storage-1`, `storage-2`,
|
||||
or `storage-3`.
|
||||
- Each storage is serviced by a Gitaly node.
|
||||
- The three Gitaly nodes store data in three separate hashed storage locations.
|
||||
- The three Gitaly nodes share data in three separate hashed storage locations.
|
||||
- The [replication factor](praefect.md#replication-factor) is `3`. There are three copies maintained
|
||||
of each repository.
|
||||
|
||||
Generally, virtual storage with Gitaly Cluster can replace direct Gitaly storage configurations, at
|
||||
the expense of additional storage needed to store each repository on multiple Gitaly nodes. The
|
||||
|
|
|
@ -7,9 +7,14 @@ type: reference
|
|||
|
||||
# Configure Gitaly Cluster **(FREE SELF)**
|
||||
|
||||
In addition to Gitaly Cluster configuration instructions available as part of
|
||||
[reference architectures](../reference_architectures/index.md) for installations for more than
|
||||
2000 users, advanced configuration instructions are available below.
|
||||
Configure Gitaly Cluster using either:
|
||||
|
||||
- The Gitaly Cluster configuration instructions available as part of
|
||||
[reference architectures](../reference_architectures/index.md) for installations for more than
|
||||
2000 users.
|
||||
- The advanced configuration instructions that follow on this page.
|
||||
|
||||
Smaller GitLab installations may need only [Gitaly itself](index.md).
|
||||
|
||||
## Requirements for configuring a Gitaly Cluster
|
||||
|
||||
|
@ -1004,7 +1009,7 @@ replication factor offers better redundancy and distribution of read workload, b
|
|||
in a higher storage cost. By default, Praefect replicates repositories to every storage in a
|
||||
virtual storage.
|
||||
|
||||
### Configure replication factors
|
||||
### Configure replication factor
|
||||
|
||||
WARNING:
|
||||
The feature is not production ready yet. After you set a replication factor, you can't unset it
|
||||
|
@ -1298,6 +1303,25 @@ sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.t
|
|||
|
||||
## Migrate to Gitaly Cluster
|
||||
|
||||
Whether migrating to Gitaly Cluster because of [NFS support deprecation](index.md#nfs-deprecation-notice)
|
||||
or to move from single Gitaly nodes, the basic process involves:
|
||||
|
||||
1. Create the required storage.
|
||||
1. Create and configure Gitaly Cluster.
|
||||
1. [Move the repositories](#move-repositories).
|
||||
|
||||
The size of the required storage can vary between instances and depends on the set
|
||||
[replication factor](#replication-factor). The migration to Gitaly Cluster might include
|
||||
implementing repository storage redundancy.
|
||||
|
||||
For a replication factor:
|
||||
|
||||
- Of `1`: NFS, Gitaly, and Gitaly Cluster have roughly the same storage requirements.
|
||||
- More than `1`: The amount of required storage is `used space * replication factor`. `used space`
|
||||
should include any planned future growth.
|
||||
|
||||
### Move Repositories
|
||||
|
||||
To migrate to Gitaly Cluster, existing repositories stored outside Gitaly Cluster must be
|
||||
moved. There is no automatic migration but the moves can be scheduled with the GitLab API.
|
||||
|
||||
|
@ -1316,11 +1340,11 @@ After creating and configuring Gitaly Cluster:
|
|||
so that the Gitaly Cluster receives all new projects. This stops new projects being created
|
||||
on existing Gitaly nodes while the migration is in progress.
|
||||
1. Schedule repository moves for:
|
||||
- [Projects](#bulk-schedule-projects).
|
||||
- [Snippets](#bulk-schedule-snippets).
|
||||
- [Groups](#bulk-schedule-groups). **(PREMIUM SELF)**
|
||||
- [Projects](#bulk-schedule-project-moves).
|
||||
- [Snippets](#bulk-schedule-snippet-moves).
|
||||
- [Groups](#bulk-schedule-group-moves). **(PREMIUM SELF)**
|
||||
|
||||
### Bulk schedule projects
|
||||
#### Bulk schedule project moves
|
||||
|
||||
1. [Schedule repository storage moves for all projects on a storage shard](../../api/project_repository_storage_moves.md#schedule-repository-storage-moves-for-all-projects-on-a-storage-shard) using the API. For example:
|
||||
|
||||
|
@ -1353,7 +1377,7 @@ After creating and configuring Gitaly Cluster:
|
|||
|
||||
1. Repeat for each storage as required.
|
||||
|
||||
### Bulk schedule snippets
|
||||
#### Bulk schedule snippet moves
|
||||
|
||||
1. [Schedule repository storage moves for all snippets on a storage shard](../../api/snippet_repository_storage_moves.md#schedule-repository-storage-moves-for-all-snippets-on-a-storage-shard) using the API. For example:
|
||||
|
||||
|
@ -1378,7 +1402,7 @@ After creating and configuring Gitaly Cluster:
|
|||
|
||||
1. Repeat for each storage as required.
|
||||
|
||||
### Bulk schedule groups **(PREMIUM SELF)**
|
||||
#### Bulk schedule group moves **(PREMIUM SELF)**
|
||||
|
||||
1. [Schedule repository storage moves for all groups on a storage shard](../../api/group_repository_storage_moves.md#schedule-repository-storage-moves-for-all-groups-on-a-storage-shard) using the API.
|
||||
|
||||
|
|
Before Width: | Height: | Size: 55 KiB |
Before Width: | Height: | Size: 34 KiB |
Before Width: | Height: | Size: 79 KiB |
Before Width: | Height: | Size: 17 KiB |
Before Width: | Height: | Size: 19 KiB |
Before Width: | Height: | Size: 28 KiB |
Before Width: | Height: | Size: 136 KiB |
|
@ -18,25 +18,20 @@ From left to right, it displays:
|
|||
- **Current Host**: the current host serving the page.
|
||||
- **Database queries**: the time taken (in milliseconds) and the total number
|
||||
of database queries, displayed in the format `00ms / 00 (00 cached) pg`. Click to display
|
||||
a modal window with more details:
|
||||
![SQL profiling using the Performance Bar](img/performance_bar_sql_queries.png)
|
||||
a modal window with more details.
|
||||
- **Gitaly calls**: the time taken (in milliseconds) and the total number of
|
||||
[Gitaly](../../gitaly/index.md) calls. Click to display a modal window with more
|
||||
details:
|
||||
![Gitaly profiling using the Performance Bar](img/performance_bar_gitaly_calls.png)
|
||||
details.
|
||||
- **Rugged calls**: the time taken (in milliseconds) and the total number of
|
||||
[Rugged](../../nfs.md#improving-nfs-performance-with-gitlab) calls.
|
||||
Click to display a modal window with more details:
|
||||
![Rugged profiling using the Performance Bar](img/performance_bar_rugged_calls.png)
|
||||
Click to display a modal window with more details.
|
||||
- **Redis calls**: the time taken (in milliseconds) and the total number of
|
||||
Redis calls. Click to display a modal window with more details:
|
||||
![Redis profiling using the Performance Bar](img/performance_bar_redis_calls.png)
|
||||
Redis calls. Click to display a modal window with more details.
|
||||
- **Elasticsearch calls**: the time taken (in milliseconds) and the total number of
|
||||
Elasticsearch calls. Click to display a modal window with more details.
|
||||
- **External HTTP calls**: the time taken (in milliseconds) and the total
|
||||
number of external calls to other systems. Click to display a modal window
|
||||
with more details
|
||||
![External call details in the Performance Bar](img/performance_bar_external_http_calls.png)
|
||||
with more details.
|
||||
- **Load timings** of the page: if your browser supports load timings (Chromium
|
||||
and Chrome) several values in milliseconds, separated by slashes.
|
||||
Click to display a modal window with more details. The values, from left to right:
|
||||
|
@ -44,8 +39,7 @@ From left to right, it displays:
|
|||
- [**First Contentful Paint**](https://web.dev/first-contentful-paint/):
|
||||
Time until something was visible to the user.
|
||||
- [**DomContentLoaded**](https://developers.google.com/web/fundamentals/performance/critical-rendering-path/measure-crp) Event.
|
||||
- **Total number of requests** the page loaded:
|
||||
![Frontend requests using the Performance Bar](img/performance_bar_frontend.png)
|
||||
- **Total number of requests** the page loaded.
|
||||
- **Trace**: If Jaeger is integrated, **Trace** links to a Jaeger tracing page
|
||||
with the current request's `correlation_id` included.
|
||||
- **+**: A link to add a request's details to the performance bar. The request
|
||||
|
@ -60,21 +54,18 @@ From left to right, it displays:
|
|||
|
||||
## Request warnings
|
||||
|
||||
Requests exceeding predefined limits display a warning **{warning}** icon and
|
||||
explanation next to the failing metric. In this example, the Gitaly call duration
|
||||
exceeded the threshold:
|
||||
Requests that exceed predefined limits display a warning **{warning}** icon and
|
||||
explanation next to the metric. In this example, the Gitaly call duration
|
||||
exceeded the threshold.
|
||||
|
||||
![Gitaly call duration exceeded threshold](img/performance_bar_gitaly_threshold.png)
|
||||
|
||||
If any requests on the current page generated warnings, the warning icon displays
|
||||
next to the **Request selector**:
|
||||
next to the **Requests** selector menu. In this selector menu, an exclamation `(!)`
|
||||
appears next to requests with warnings.
|
||||
|
||||
![Request selector showing two requests with warnings](img/performance_bar_request_selector_warning.png)
|
||||
|
||||
Requests with warnings display `(!)` after their path in the **Request selector**:
|
||||
|
||||
![Request selector showing dropdown](img/performance_bar_request_selector_warning_expanded.png)
|
||||
|
||||
## Enable the Performance Bar via the Admin Area
|
||||
|
||||
The GitLab Performance Bar is disabled by default. To enable it for a given group:
|
||||
|
|
|
@ -464,7 +464,7 @@ Some analyzers make it possible to filter out vulnerabilities under a given thre
|
|||
|
||||
| CI/CD variable | Default value | Description |
|
||||
|------------------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| `SAST_EXCLUDED_PATHS` | `spec, test, tests, tmp` | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec` ). Parent directories also match patterns. You might need to exclude temporary directories used by your build tool as these can generate false positives. |
|
||||
| `SAST_EXCLUDED_PATHS` | `spec, test, tests, tmp` | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. You might need to exclude temporary directories used by your build tool as these can generate false positives. To exclude paths, copy and paste the default excluded paths, then **add** your own paths to be excluded. If you don't specify the default excluded paths, you will override the defaults and _only_ paths you specify will be excluded from the SAST scans. |
|
||||
| `SEARCH_MAX_DEPTH` | 4 | SAST searches the repository to detect the programming languages used, and selects the matching analyzers. Set the value of `SEARCH_MAX_DEPTH` to specify how many directory levels the search phase should span. After the analyzers have been selected, the _entire_ repository is analyzed. |
|
||||
| `SAST_BANDIT_EXCLUDED_PATHS` | | Comma-separated list of paths to exclude from scan. Uses Python's [`fnmatch` syntax](https://docs.python.org/2/library/fnmatch.html); For example: `'*/tests/*, */venv/*'` |
|
||||
| `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. |
|
||||
|
|