kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Switch to gitlab-workhorse

Browse source
This commit is contained in:
Jacob Vosmaer 2015-10-29 14:21:24 +01:00
parent 3c23b6515f
commit c5132e94e1
8 changed files with 249 additions and 65 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
GITLAB_GIT_HTTP_SERVER_VERSION
View file

@ -1 +0,0 @@
0.3.0

1
GITLAB_WORKHORSE_VERSION Normal file
View file

@ -0,0 +1 @@
0.3.1

187
doc/update/8.1-to-8.2.md Normal file
View file

@ -0,0 +1,187 @@
# From 8.1 to 8.2
**NOTE:** GitLab 8.0 introduced several significant changes related to
installation and configuration which *are not duplicated here*. Be sure you're
already running a working version of 8.0 before proceeding with this guide.
### 0. Double-check your Git version
**This notice applies only to /usr/local/bin/git**
If you compiled Git from source on your GitLab server then please double-check
that you are using a version that protects against CVE-2014-9390. For six
months after this vulnerability became known the GitLab installation guide
still contained instructions that would install the outdated, 'vulnerable' Git
version 2.1.2.
Run the following command to get your current Git version:
```sh
/usr/local/bin/git --version
```
If you see 'No such file or directory' then you did not install Git according
to the outdated instructions from the GitLab installation guide and you can go
to the next step 'Stop server' below.
If you see a version string then it should be v1.8.5.6, v1.9.5, v2.0.5, v2.1.4,
v2.2.1 or newer. You can use the [instructions in the GitLab source
installation
guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#1-packages-dependencies)
to install a newer version of Git.
### 1. Stop server
sudo service gitlab stop
### 2. Backup
```bash
cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
```
### 3. Get latest code
```bash
sudo -u git -H git fetch --all
sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically
```
For GitLab Community Edition:
```bash
sudo -u git -H git checkout 8-2-stable
```
OR
For GitLab Enterprise Edition:
```bash
sudo -u git -H git checkout 8-2-stable-ee
```
### 4. Update gitlab-shell
```bash
cd /home/git/gitlab-shell
sudo -u git -H git fetch
sudo -u git -H git checkout v2.6.5
```
### 5. Replace gitlab-git-http-server with gitlab-workhorse
Install and compile gitlab-workhorse. This requires [Go
1.5](https://golang.org/dl) which should already be on your system
from GitLab 8.1.
```bash
cd /home/git
sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git
sudo -u git -H git checkout 0.3.1
sudo -u git -H make
```
Update the GitLab init script and 'default' file.
```
cd /home/git/gitlab
sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
test -e /etc/default/gitlab && \
sudo sed -i .pre-8.2 's/^\([^=]*\)gitlab_git_http_server/\1gitlab_workhorse/' /etc/default/gitlab
```
Make sure that you also update your **NGINX configuration** to use
the new gitlab-workhorse.socket file.
### 6. Install libs, migrations, etc.
```bash
cd /home/git/gitlab
# MySQL installations (note: the line below states '--without postgres')
sudo -u git -H bundle install --without postgres development test --deployment
# PostgreSQL installations (note: the line below states '--without mysql')
sudo -u git -H bundle install --without mysql development test --deployment
# Run database migrations
sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
# Clean up assets and cache
sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production
# Update init.d script
sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
```
### 7. Update configuration files
#### New configuration options for `gitlab.yml`
There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`:
```sh
git diff origin/8-1-stable:config/gitlab.yml.example origin/8-2-stable:config/gitlab.yml.example
```
#### Nginx configuration
View changes between the previous recommended Nginx configuration and the
current one:
```sh
# For HTTPS configurations
git diff origin/8-1-stable:lib/support/nginx/gitlab-ssl origin/8-2-stable:lib/support/nginx/gitlab-ssl
# For HTTP configurations
git diff origin/8-1-stable:lib/support/nginx/gitlab origin/8-2-stable:lib/support/nginx/gitlab
```
If you are using Apache instead of NGINX please see the updated [Apache templates].
Also note that because Apache does not support upstreams behind Unix sockets you
will need to let gitlab-git-http-server listen on a TCP port. You can do this
via [/etc/default/gitlab].
[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache
[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-2-stable/lib/support/init.d/gitlab.default.example#L34
### 8. Start application
sudo service gitlab start
sudo service nginx restart
### 9. Check application status
Check if GitLab and its environment are configured correctly:
sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
To make sure you didn't miss anything run a more thorough check:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
If all items are green, then congratulations, the upgrade is complete!
## Things went south? Revert to previous version (8.0)
### 1. Revert the code to the previous version
Follow the [upgrade guide from 7.14 to 8.0](7.14-to-8.0.md), except for the database migration
(The backup is already migrated to the previous version)
### 2. Restore from the backup
```bash
cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production
```
If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above.
## Troubleshooting
### "You appear to have cloned an empty repository."
See the [7.14 to 8.0 update guide](7.14-to-8.0.md#troubleshooting).

2
lib/gitlab/backend/grack_auth.rb
View file

@ -34,7 +34,7 @@ module Grack
auth!
if project && authorized_request?
# Tell gitlab-git-http-server the request is OK, and what the GL_ID is
# Tell gitlab-workhorse the request is OK, and what the GL_ID is
render_grack_auth_ok
elsif @user.nil? && !@ci
unauthorized

68
lib/support/init.d/gitlab
View file

@ -37,10 +37,9 @@ web_server_pid_path="$pid_path/unicorn.pid"
sidekiq_pid_path="$pid_path/sidekiq.pid"
mail_room_enabled=false
mail_room_pid_path="$pid_path/mail_room.pid"
gitlab_git_http_server_pid_path="$pid_path/gitlab-git-http-server.pid"
gitlab_git_http_server_options="-listenUmask 0 -listenNetwork unix -listenAddr $socket_path/gitlab-git-http-server.socket -authBackend http://127.0.0.1:8080"
gitlab_git_http_server_repo_root='/home/git/repositories'
gitlab_git_http_server_log="$app_root/log/gitlab-git-http-server.log"
gitlab_workhorse_pid_path="$pid_path/gitlab-workhorse.pid"
gitlab_workhorse_options="-listenUmask 0 -listenNetwork unix -listenAddr $socket_path/gitlab-workhorse.socket -authBackend http://127.0.0.1:8080"
gitlab_workhorse_log="$app_root/log/gitlab-workhorse.log"
shell_path="/bin/bash"
# Read configuration variable file if it is present
@ -76,8 +75,8 @@ check_pids(){
else
spid=0
fi
if [ -f "$gitlab_git_http_server_pid_path" ]; then
hpid=$(cat "$gitlab_git_http_server_pid_path")
if [ -f "$gitlab_workhorse_pid_path" ]; then
hpid=$(cat "$gitlab_workhorse_pid_path")
else
hpid=0
fi
@ -94,7 +93,7 @@ check_pids(){
wait_for_pids(){
# We are sleeping a bit here mostly because sidekiq is slow at writing it's pid
i=0;
while [ ! -f $web_server_pid_path ] || [ ! -f $sidekiq_pid_path ] || [ ! -f $gitlab_git_http_server_pid_path ] || { [ "$mail_room_enabled" = true ] && [ ! -f $mail_room_pid_path ]; }; do
while [ ! -f $web_server_pid_path ] || [ ! -f $sidekiq_pid_path ] || [ ! -f $gitlab_workhorse_pid_path ] || { [ "$mail_room_enabled" = true ] && [ ! -f $mail_room_pid_path ]; }; do
sleep 0.1;
i=$((i+1))
if [ $((i%10)) = 0 ]; then
@ -131,9 +130,9 @@ check_status(){
fi
if [ $hpid -ne 0 ]; then
kill -0 "$hpid" 2>/dev/null
gitlab_git_http_server_status="$?"
gitlab_workhorse_status="$?"
else
gitlab_git_http_server_status="-1"
gitlab_workhorse_status="-1"
fi
if [ "$mail_room_enabled" = true ]; then
if [ $mpid -ne 0 ]; then
@ -143,7 +142,7 @@ check_status(){
mail_room_status="-1"
fi
fi
if [ $web_status = 0 ] && [ $sidekiq_status = 0 ] && [ $gitlab_git_http_server_status = 0 ] && { [ "$mail_room_enabled" != true ] || [ $mail_room_status = 0 ]; }; then
if [ $web_status = 0 ] && [ $sidekiq_status = 0 ] && [ $gitlab_workhorse_status = 0 ] && { [ "$mail_room_enabled" != true ] || [ $mail_room_status = 0 ]; }; then
gitlab_status=0
else
# http://refspecs.linuxbase.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/iniscrptact.html
@ -171,9 +170,9 @@ check_stale_pids(){
exit 1
fi
fi
if [ "$hpid" != "0" ] && [ "$gitlab_git_http_server_status" != "0" ]; then
echo "Removing stale gitlab-git-http-server pid. This is most likely caused by gitlab-git-http-server crashing the last time it ran."
if ! rm "$gitlab_git_http_server_pid_path"; then
if [ "$hpid" != "0" ] && [ "$gitlab_workhorse_status" != "0" ]; then
echo "Removing stale gitlab-workhorse pid. This is most likely caused by gitlab-workhorse crashing the last time it ran."
if ! rm "$gitlab_workhorse_pid_path"; then
echo "Unable to remove stale pid, exiting"
exit 1
fi
@ -190,7 +189,7 @@ check_stale_pids(){
## If no parts of the service is running, bail out.
exit_if_not_running(){
check_stale_pids
if [ "$web_status" != "0" ] && [ "$sidekiq_status" != "0" ] && [ "$gitlab_git_http_server_status" != "0" ] && { [ "$mail_room_enabled" != true ] || [ "$mail_room_status" != "0" ]; }; then
if [ "$web_status" != "0" ] && [ "$sidekiq_status" != "0" ] && [ "$gitlab_workhorse_status" != "0" ] && { [ "$mail_room_enabled" != true ] || [ "$mail_room_status" != "0" ]; }; then
echo "GitLab is not running."
exit
fi
@ -206,8 +205,8 @@ start_gitlab() {
if [ "$sidekiq_status" != "0" ]; then
echo "Starting GitLab Sidekiq"
fi
if [ "$gitlab_git_http_server_status" != "0" ]; then
echo "Starting gitlab-git-http-server"
if [ "$gitlab_workhorse_status" != "0" ]; then
echo "Starting gitlab-workhorse"
fi
if [ "$mail_room_enabled" = true ] && [ "$mail_room_status" != "0" ]; then
echo "Starting GitLab MailRoom"
@ -230,15 +229,14 @@ start_gitlab() {
RAILS_ENV=$RAILS_ENV bin/background_jobs start &
fi
if [ "$gitlab_git_http_server_status" = "0" ]; then
echo "The gitlab-git-http-server is already running with pid $spid, not restarting"
if [ "$gitlab_workhorse_status" = "0" ]; then
echo "The gitlab-workhorse is already running with pid $spid, not restarting"
else
# No need to remove a socket, gitlab-git-http-server does this itself
$app_root/bin/daemon_with_pidfile $gitlab_git_http_server_pid_path \
$app_root/../gitlab-git-http-server/gitlab-git-http-server \
$gitlab_git_http_server_options \
$gitlab_git_http_server_repo_root \
>> $gitlab_git_http_server_log 2>&1 &
# No need to remove a socket, gitlab-workhorse does this itself
$app_root/bin/daemon_with_pidfile $gitlab_workhorse_pid_path \
$app_root/../gitlab-workhorse/gitlab-workhorse \
$gitlab_workhorse_options \
>> $gitlab_workhorse_log 2>&1 &
fi
if [ "$mail_room_enabled" = true ]; then
@ -268,9 +266,9 @@ stop_gitlab() {
echo "Shutting down GitLab Sidekiq"
RAILS_ENV=$RAILS_ENV bin/background_jobs stop
fi
if [ "$gitlab_git_http_server_status" = "0" ]; then
echo "Shutting down gitlab-git-http-server"
kill -- $(cat $gitlab_git_http_server_pid_path)
if [ "$gitlab_workhorse_status" = "0" ]; then
echo "Shutting down gitlab-workhorse"
kill -- $(cat $gitlab_workhorse_pid_path)
fi
if [ "$mail_room_enabled" = true ] && [ "$mail_room_status" = "0" ]; then
echo "Shutting down GitLab MailRoom"
@ -278,11 +276,11 @@ stop_gitlab() {
fi
# If something needs to be stopped, lets wait for it to stop. Never use SIGKILL in a script.
while [ "$web_status" = "0" ] || [ "$sidekiq_status" = "0" ] || [ "$gitlab_git_http_server_status" = "0" ] || { [ "$mail_room_enabled" = true ] && [ "$mail_room_status" = "0" ]; }; do
while [ "$web_status" = "0" ] || [ "$sidekiq_status" = "0" ] || [ "$gitlab_workhorse_status" = "0" ] || { [ "$mail_room_enabled" = true ] && [ "$mail_room_status" = "0" ]; }; do
sleep 1
check_status
printf "."
if [ "$web_status" != "0" ] && [ "$sidekiq_status" != "0" ] && [ "$gitlab_git_http_server_status" != "0" ] && { [ "$mail_room_enabled" != true ] || [ "$mail_room_status" != "0" ]; }; then
if [ "$web_status" != "0" ] && [ "$sidekiq_status" != "0" ] && [ "$gitlab_workhorse_status" != "0" ] && { [ "$mail_room_enabled" != true ] || [ "$mail_room_status" != "0" ]; }; then
printf "\n"
break
fi
@ -292,7 +290,7 @@ stop_gitlab() {
# Cleaning up unused pids
rm "$web_server_pid_path" 2>/dev/null
# rm "$sidekiq_pid_path" 2>/dev/null # Sidekiq seems to be cleaning up it's own pid.
rm -f "$gitlab_git_http_server_pid_path"
rm -f "$gitlab_workhorse_pid_path"
if [ "$mail_room_enabled" = true ]; then
rm "$mail_room_pid_path" 2>/dev/null
fi
@ -303,7 +301,7 @@ stop_gitlab() {
## Prints the status of GitLab and it's components.
print_status() {
check_status
if [ "$web_status" != "0" ] && [ "$sidekiq_status" != "0" ] && [ "$gitlab_git_http_server_status" != "0" ] && { [ "$mail_room_enabled" != true ] || [ "$mail_room_status" != "0" ]; }; then
if [ "$web_status" != "0" ] && [ "$sidekiq_status" != "0" ] && [ "$gitlab_workhorse_status" != "0" ] && { [ "$mail_room_enabled" != true ] || [ "$mail_room_status" != "0" ]; }; then
echo "GitLab is not running."
return
fi
@ -317,10 +315,10 @@ print_status() {
else
printf "The GitLab Sidekiq job dispatcher is \033[31mnot running\033[0m.\n"
fi
if [ "$gitlab_git_http_server_status" = "0" ]; then
echo "The gitlab-git-http-server with pid $hpid is running."
if [ "$gitlab_workhorse_status" = "0" ]; then
echo "The gitlab-workhorse with pid $hpid is running."
else
printf "The gitlab-git-http-server is \033[31mnot running\033[0m.\n"
printf "The gitlab-workhorse is \033[31mnot running\033[0m.\n"
fi
if [ "$mail_room_enabled" = true ]; then
if [ "$mail_room_status" = "0" ]; then
@ -360,7 +358,7 @@ reload_gitlab(){
## Restarts Sidekiq and Unicorn.
restart_gitlab(){
check_status
if [ "$web_status" = "0" ] || [ "$sidekiq_status" = "0" ] || [ "$gitlab_git_http_server" = "0" ] || { [ "$mail_room_enabled" = true ] && [ "$mail_room_status" = "0" ]; }; then
if [ "$web_status" = "0" ] || [ "$sidekiq_status" = "0" ] || [ "$gitlab_workhorse" = "0" ] || { [ "$mail_room_enabled" = true ] && [ "$mail_room_status" = "0" ]; }; then
stop_gitlab
fi
start_gitlab

11
lib/support/init.d/gitlab.default.example
View file

@ -30,15 +30,14 @@ web_server_pid_path="$pid_path/unicorn.pid"
# The default is "$pid_path/sidekiq.pid"
sidekiq_pid_path="$pid_path/sidekiq.pid"
gitlab_git_http_server_pid_path="$pid_path/gitlab-git-http-server.pid"
# The -listenXxx settings determine where gitlab-git-http-server
gitlab_workhorse_pid_path="$pid_path/gitlab-workhorse.pid"
# The -listenXxx settings determine where gitlab-workhorse
# listens for connections from NGINX. To listen on localhost:8181, write
# '-listenNetwork tcp -listenAddr localhost:8181'.
# The -authBackend setting tells gitlab-git-http-server where it can reach
# The -authBackend setting tells gitlab-workhorse where it can reach
# Unicorn.
gitlab_git_http_server_options="-listenUmask 0 -listenNetwork unix -listenAddr $socket_path/gitlab-git-http-server.socket -authBackend http://127.0.0.1:8080"
gitlab_git_http_server_repo_root="/home/git/repositories"
gitlab_git_http_server_log="$app_root/log/gitlab-git-http-server.log"
gitlab_workhorse_options="-listenUmask 0 -listenNetwork unix -listenAddr $socket_path/gitlab-workhorse.socket -authBackend http://127.0.0.1:8080"
gitlab_workhorse_log="$app_root/log/gitlab-workhorse.log"
# mail_room_enabled specifies whether mail_room, which is used to process incoming email, is enabled.
# This is required for the Reply by email feature.

22
lib/support/nginx/gitlab
View file

@ -38,8 +38,8 @@ upstream gitlab {
server unix:/home/git/gitlab/tmp/sockets/gitlab.socket fail_timeout=0;
}
upstream gitlab-git-http-server {
server unix:/home/git/gitlab/tmp/sockets/gitlab-git-http-server.socket fail_timeout=0;
upstream gitlab-workhorse {
server unix:/home/git/gitlab/tmp/sockets/gitlab-workhorse.socket fail_timeout=0;
}
## Normal HTTP host
@ -114,24 +114,24 @@ server {
}
location ~ ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$ {
# 'Error' 418 is a hack to re-use the @gitlab-git-http-server block
error_page 418 = @gitlab-git-http-server;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/[\w\.-]+/[\w\.-]+/repository/archive {
# 'Error' 418 is a hack to re-use the @gitlab-git-http-server block
error_page 418 = @gitlab-git-http-server;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/api/v3/projects/.*/repository/archive {
# 'Error' 418 is a hack to re-use the @gitlab-git-http-server block
error_page 418 = @gitlab-git-http-server;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location @gitlab-git-http-server {
location @gitlab-workhorse {
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
# gzip off;
@ -147,7 +147,7 @@ server {
# The following settings only work with NGINX 1.7.11 or newer
#
# # Pass chunked request bodies to gitlab-git-http-server as-is
# # Pass chunked request bodies to gitlab-workhorse as-is
# proxy_request_buffering off;
# proxy_http_version 1.1;
@ -156,7 +156,7 @@ server {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://gitlab-git-http-server;
proxy_pass http://gitlab-workhorse;
}
## Enable gzip compression as per rails guide:

22
lib/support/nginx/gitlab-ssl
View file

@ -42,8 +42,8 @@ upstream gitlab {
server unix:/home/git/gitlab/tmp/sockets/gitlab.socket fail_timeout=0;
}
upstream gitlab-git-http-server {
server unix:/home/git/gitlab/tmp/sockets/gitlab-git-http-server.socket fail_timeout=0;
upstream gitlab-workhorse {
server unix:/home/git/gitlab/tmp/sockets/gitlab-workhorse.socket fail_timeout=0;
}
## Redirects all HTTP traffic to the HTTPS host
@ -161,24 +161,24 @@ server {
}
location ~ ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$ {
# 'Error' 418 is a hack to re-use the @gitlab-git-http-server block
error_page 418 = @gitlab-git-http-server;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/[\w\.-]+/[\w\.-]+/repository/archive {
# 'Error' 418 is a hack to re-use the @gitlab-git-http-server block
error_page 418 = @gitlab-git-http-server;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/api/v3/projects/.*/repository/archive {
# 'Error' 418 is a hack to re-use the @gitlab-git-http-server block
error_page 418 = @gitlab-git-http-server;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location @gitlab-git-http-server {
location @gitlab-workhorse {
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
gzip off;
@ -194,7 +194,7 @@ server {
# The following settings only work with NGINX 1.7.11 or newer
#
# # Pass chunked request bodies to gitlab-git-http-server as-is
# # Pass chunked request bodies to gitlab-workhorse as-is
# proxy_request_buffering off;
# proxy_http_version 1.1;
@ -203,7 +203,7 @@ server {
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://gitlab-git-http-server;
proxy_pass http://gitlab-workhorse;
}
## Enable gzip compression as per rails guide: