kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Add documentation about Gitaly concurrency limiter

Browse source
This commit is contained in:
Jacob Vosmaer 2019-08-26 20:51:15 +00:00 committed by Achilleas Pipinellis
parent 67ac55cccc
commit 5095e0cfe3
1 changed files with 41 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

41
doc/administration/gitaly/index.md
View file

@ -518,6 +518,47 @@ One current feature of GitLab that still requires a shared directory (NFS) is
There is [work in progress](https://gitlab.com/gitlab-org/gitlab-pages/issues/196) There is [work in progress](https://gitlab.com/gitlab-org/gitlab-pages/issues/196)
to eliminate the need for NFS to support GitLab Pages. to eliminate the need for NFS to support GitLab Pages.
## Limiting RPC concurrency
It can happen that CI clone traffic puts a large strain on your Gitaly
service. The bulk of the work gets done in the SSHUploadPack (for Git
SSH) and PostUploadPack (for Git HTTP) RPC's. To prevent such workloads
from overcrowding your Gitaly server you can set concurrency limits in
Gitaly's configuration file.
```ruby
# in /etc/gitlab/gitlab.rb
gitaly['concurrency'] = [
{
'rpc' => "/gitaly.SmartHTTPService/PostUploadPack",
'max_per_repo' => 20
},
{
'rpc' => "/gitaly.SSHService/SSHUploadPack",
'max_per_repo' => 20
}
]
```
This will limit the number of in-flight RPC calls for the given RPC's.
The limit is applied per repository. In the example above, each on the
Gitaly server can have at most 20 simultaneous PostUploadPack calls in
flight, and the same for SSHUploadPack. If another request comes in for
a repository that hase used up its 20 slots, that request will get
queued.
You can observe the behavior of this queue via the Gitaly logs and via
Prometheus. In the Gitaly logs, you can look for the string (or
structured log field) `acquire_ms`. Messages that have this field are
reporting about the concurrency limiter. In Prometheus, look for the
`gitaly_rate_limiting_in_progress`, `gitaly_rate_limiting_queued` and
`gitaly_rate_limiting_seconds` metrics.
The name of the Prometheus metric is not quite right because this is a
concurrency limiter, not a rate limiter. If a client makes 1000 requests
in a row in a very short timespan, the concurrency will not exceed 1,
and this mechanism (the concurrency limiter) will do nothing.
## Troubleshooting Gitaly ## Troubleshooting Gitaly
### Commits, pushes, and clones return a 401 ### Commits, pushes, and clones return a 401