gitlab-org--gitlab-foss/doc/administration/high_availability/sidekiq.md

type
reference

Configuring Sidekiq

This section discusses how to configure an external Sidekiq instance.

Sidekiq requires connection to the Redis, PostgreSQL and Gitaly instance. To configure the Sidekiq node:

1. SSH into the Sidekiq server.

2. Download/install the Omnibus GitLab package you want using steps 1 and 2 from the GitLab downloads page. Do not complete any other steps on the download page.

3. Open /etc/gitlab/gitlab.rb with your editor.

4. Generate the Sidekiq configuration:

   sidekiq['listen_address'] = "10.10.1.48"
   
   ## Optional: Enable extra Sidekiq processes
   sidekiq_cluster['enable'] = true
   sidekiq_cluster['enable'] = true
     "elastic_indexer"
   ]
   

5. Setup Sidekiq's connection to Redis:

   ## Must be the same in every sentinel node
   redis['master_name'] = 'gitlab-redis'
   
   ## The same password for Redis authentication you set up for the master node.
   redis['master_password'] = 'YOUR_PASSOWORD'
   
   ## A list of sentinels with `host` and `port`
   gitlab_rails['redis_sentinels'] = [
       {'host' => '10.10.1.34', 'port' => 26379},
       {'host' => '10.10.1.35', 'port' => 26379},
       {'host' => '10.10.1.36', 'port' => 26379},
     ]
   

6. Setup Sidekiq's connection to Gitaly:

   git_data_dirs({
     'default' => { 'gitaly_address' => 'tcp://gitaly:8075' },
   })
   gitlab_rails['gitaly_token'] = 'YOUR_TOKEN'
   

7. Setup Sidekiq's connection to PostgreSQL:

   gitlab_rails['db_host'] = '10.10.1.30'
   gitlab_rails['db_password'] = 'YOUR_PASSOWORD'
   gitlab_rails['db_port'] = '5432'
   gitlab_rails['db_adapter'] = 'postgresql'
   gitlab_rails['db_encoding'] = 'unicode'
   gitlab_rails['auto_migrate'] = false
   

   Remember to add the Sidekiq nodes to the PostgreSQL whitelist:

   postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.10.1.30/32 10.10.1.31/32 10.10.1.32/32 10.10.1.33/32 10.10.1.38/32)
   

8. Disable other services:

   nginx['enable'] = false
   grafana['enable'] = false
   prometheus['enable'] = false
   gitlab_rails['auto_migrate'] = false
   alertmanager['enable'] = false
   gitaly['enable'] = false
   gitlab_monitor['enable'] = false
   gitlab_workhorse['enable'] = false
   nginx['enable'] = false
   postgres_exporter['enable'] = false
   postgresql['enable'] = false
   redis['enable'] = false
   redis_exporter['enable'] = false
   puma['enable'] = false
   gitlab_exporter['enable'] = false
   

9. Run gitlab-ctl reconfigure.

NOTE: Note: You will need to restart the Sidekiq nodes after an update has occurred and database migrations performed.

Example configuration

Here's what the ending /etc/gitlab/gitlab.rb would look like:

########################################
#####        Services Disabled       ###
########################################

nginx['enable'] = false
grafana['enable'] = false
prometheus['enable'] = false
gitlab_rails['auto_migrate'] = false
alertmanager['enable'] = false
gitaly['enable'] = false
gitlab_monitor['enable'] = false
gitlab_workhorse['enable'] = false
nginx['enable'] = false
postgres_exporter['enable'] = false
postgresql['enable'] = false
redis['enable'] = false
redis_exporter['enable'] = false
puma['enable'] = false
gitlab_exporter['enable'] = false

########################################
####              Redis              ###
########################################

## Must be the same in every sentinel node
redis['master_name'] = 'gitlab-redis'

## The same password for Redis authentication you set up for the master node.
redis['master_password'] = 'YOUR_PASSOWORD'

## A list of sentinels with `host` and `port`
gitlab_rails['redis_sentinels'] = [
    {'host' => '10.10.1.34', 'port' => 26379},
    {'host' => '10.10.1.35', 'port' => 26379},
    {'host' => '10.10.1.36', 'port' => 26379},
  ]

#######################################
###              Gitaly             ###
#######################################

git_data_dirs({
  'default' => { 'gitaly_address' => 'tcp://gitaly:8075' },
})
gitlab_rails['gitaly_token'] = 'YOUR_TOKEN'

#######################################
###            Postgres             ###
#######################################
gitlab_rails['db_host'] = '10.10.1.30'
gitlab_rails['db_password'] = 'YOUR_PASSOWORD'
gitlab_rails['db_port'] = '5432'
gitlab_rails['db_adapter'] = 'postgresql'
gitlab_rails['db_encoding'] = 'unicode'
gitlab_rails['auto_migrate'] = false

#######################################
###      Sidekiq configuration      ###
#######################################
sidekiq['listen_address'] = "10.10.1.48"

#######################################
###     Monitoring configuration    ###
#######################################
consul['enable'] = true
consul['monitoring_service_discovery'] =  true

consul['configuration'] = {
  bind_addr: '10.10.1.48',
  retry_join: %w(10.10.1.34 10.10.1.35 10.10.1.36)
}

# Set the network addresses that the exporters will listen on
node_exporter['listen_address'] = '10.10.1.48:9100'

# Rails Status for prometheus
gitlab_rails['monitoring_whitelist'] = ['10.10.1.42', '127.0.0.1']

Further reading

Related Sidekiq configuration:

1. Extra Sidekiq processes
2. Using the GitLab-Sidekiq chart