gitlab-org--gitlab-foss/doc/administration/gitaly/praefect.md

Praefect

NOTE: Note: Praefect is an experimental service, and for testing purposes only at this time.

Omnibus

Architecture

For this document, the following network topology is assumed:

graph TB
  GitLab --> Gitaly;
  GitLab --> Praefect;
  Praefect --> Preafect-Git-1;
  Praefect --> Preafect-Git-2;
  Praefect --> Preafect-Git-3;

Where GitLab is the collection of clients that can request Git operations. Gitaly is a Gitaly server before using Praefect. The Praefect node has two storage nodes attached. Praefect itself doesn't storage data, but connects to three Gitaly nodes, Praefect-Git-1, Praefect-Git-2, and Praefect-Git-3. There should be no knowledge other than with Praefect about the existence of the Praefect-Git-X nodes.

Setup

In this setup guide, the Gitaly node will be added first, then Praefect, and lastly we update the GitLab configuration.

Gitaly

In their own machine, configure the Gitaly server as described in the gitaly documentation.

Praefect

Next, Praefect has to be enabled on its own node. Disable all other services, and add each Gitaly node that will be connected to Praefect. In the example below, the Gitaly nodes are named praefect-git-X. Note that one node is designated as primary, by setting the primary to true:

# /etc/gitlab/gitlab.rb

# Avoid running unnecessary services on the Gitaly server
postgresql['enable'] = false
redis['enable'] = false
nginx['enable'] = false
prometheus['enable'] = false
unicorn['enable'] = false
sidekiq['enable'] = false
gitlab_workhorse['enable'] = false
gitaly['enable'] = false

praefect['enable'] = true
praefect['storage_nodes'] = [
  {
    'storage' => 'praefect-git-1',
    'address' => 'tcp://praefect-git-1.internal',
    'primary' => true
  },
  {
    'storage' => 'praefect-git-2',
    'address' => 'tcp://praefect-git-2.internal'
  },
  {
    'storage' => 'praefect-git-3',
    'address' => 'tcp://praefect-git-3.internal'
  }
]

Save the file and reconfigure Praefect.

GitLab

When Praefect is running, it should be exposed as a storage to GitLab. This is done through setting the git_data_dirs. Assuming the default storage configuration is used, there would be two storages available to GitLab:

git_data_dirs({
  "default" => {
    "gitaly_address" => "tcp://gitaly.internal"
  },
  "praefect" => {
    "gitaly_address" => "tcp://praefect.internal:2305"
  }
})

Restart GitLab using gitlab-ctl restart on the GitLab node.