kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity
gitlab-org--gitlab-foss/doc/update/mysql_to_postgresql.md
Evan Read eb86630959 Ensure all lists are surrounded by new lines 
Markdown renderers find it easier to determine
where lists start and end when lists are surrounded
by new lines.

For consistency, also ensure entries in the list
are aligned when they span multipls lines.
2019-02-22 13:17:10 +00:00

286 lines
9.5 KiB
Markdown
Raw Blame History

---
last_updated: 2018-02-07
---
# Migrating from MySQL to PostgreSQL
> **Note:** This guide assumes you have a working GitLab instance with
> MySQL and want to migrate to bundled PostgreSQL database.
## Omnibus installation
### Prerequisites
First, we'll need to enable the bundled PostgreSQL database with up-to-date
schema. Next, we'll use [pgloader](http://pgloader.io) to migrate the data
from the old MySQL database to the new PostgreSQL one.
Here's what you'll need to have installed:
- pgloader 3.4.1+
- Omnibus GitLab
- MySQL
### Enable bundled PostgreSQL database
1. Stop GitLab:
``` bash
sudo gitlab-ctl stop
```
1. Edit `/etc/gitlab/gitlab.rb` to enable bundled PostgreSQL:
```
postgresql['enable'] = true
```
1. Edit `/etc/gitlab/gitlab.rb` to use the bundled PostgreSQL. Please check
all the settings beginning with `db_`, such as `gitlab_rails['db_adapter']`
and alike. You could just comment all of them out so that we'll just use
the defaults.
1. [Reconfigure GitLab] for the changes to take effect:
``` bash
sudo gitlab-ctl reconfigure
```
1. Start Unicorn and PostgreSQL so that we can prepare the schema:
``` bash
sudo gitlab-ctl start unicorn
sudo gitlab-ctl start postgresql
```
1. Run the following commands to prepare the schema:
``` bash
sudo gitlab-rake db:create db:migrate
```
1. Stop Unicorn to prevent other database access from interfering with the loading of data:
``` bash
sudo gitlab-ctl stop unicorn
```
After these steps, you'll have a fresh PostgreSQL database with up-to-date schema.
### Migrate data from MySQL to PostgreSQL
Now, you can use pgloader to migrate the data from MySQL to PostgreSQL:
1. Save the following snippet in a `commands.load` file, and edit with your
database `username`, `password` and `host`:
```
LOAD DATABASE
FROM mysql://username:password@host/gitlabhq_production
INTO postgresql://gitlab-psql@unix://var/opt/gitlab/postgresql:/gitlabhq_production
WITH include no drop, truncate, disable triggers, create no tables,
create no indexes, preserve index names, no foreign keys,
data only
ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public'
;
```
1. Start the migration:
``` bash
sudo -u gitlab-psql pgloader commands.load
```
1. Once the migration finishes, you should see a summary table that looks like
the following:
```
table name read imported errors total time
----------------------------------------------- --------- --------- --------- --------------
fetch meta data 119 119 0 0.388s
Truncate 119 119 0 1.134s
----------------------------------------------- --------- --------- --------- --------------
public.abuse_reports 0 0 0 0.490s
public.appearances 0 0 0 0.488s
.
.
.
public.web_hook_logs 0 0 0 1.080s
----------------------------------------------- --------- --------- --------- --------------
COPY Threads Completion 4 4 0 2.008s
Reset Sequences 113 113 0 0.304s
Install Comments 0 0 0 0.000s
----------------------------------------------- --------- --------- --------- --------------
Total import time 1894 1894 0 12.497s
```
If there is no output for more than 30 minutes, it's possible pgloader encountered an error. See
the [troubleshooting guide](#Troubleshooting) for more details.
1. Start GitLab:
``` bash
sudo gitlab-ctl start
```
Now, you can verify that everything worked by visiting GitLab.
### Troubleshooting
#### Permissions
Note that the PostgreSQL user that you use for the above MUST have **superuser** privileges. Otherwise, you may see
a similar message to the following:
```
debugger invoked on a CL-POSTGRES-ERROR:INSUFFICIENT-PRIVILEGE in thread
#<THREAD "lparallel" RUNNING {10078A3513}>:
Database error 42501: permission denied: "RI_ConstraintTrigger_a_20937" is a system trigger
QUERY: ALTER TABLE ci_builds DISABLE TRIGGER ALL;
2017-08-23T00:36:56.782000Z ERROR Database error 42501: permission denied: "RI_ConstraintTrigger_c_20864" is a system trigger
QUERY: ALTER TABLE approver_groups DISABLE TRIGGER ALL;
```
#### Experiencing 500 errors after the migration
If you experience 500 errors after the migration, try to clear the cache:
``` bash
sudo gitlab-rake cache:clear
```
[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure
## Source installation
### Prerequisites
#### Install PostgreSQL and create database
See [installation guide](../install/installation.md#6-database).
#### Install [pgloader](http://pgloader.io) 3.4.1+
Install directly from your distro:
``` bash
sudo apt-get install pgloader
```
If this version is too old, use PostgreSQL's repository:
``` bash
# add repository
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
# add key
sudo apt-get install wget ca-certificates
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
# install package
sudo apt-get update
sudo apt-get install pgloader
```
### Enable bundled PostgreSQL database
1. Stop GitLab:
``` bash
sudo service gitlab stop
```
1. Switch database from MySQL to PostgreSQL
``` bash
cd /home/git/gitlab
sudo -u git mv config/database.yml config/database.yml.bak
sudo -u git cp config/database.yml.postgresql config/database.yml
sudo -u git -H chmod o-rwx config/database.yml
```
1. Install Gems related to Postgresql
``` bash
sudo -u git -H rm .bundle/config
sudo -u git -H bundle install --deployment --without development test mysql aws kerberos
```
1. Run the following commands to prepare the schema:
``` bash
sudo -u git -H bundle exec rake db:create db:migrate RAILS_ENV=production
```
After these steps, you'll have a fresh PostgreSQL database with up-to-date schema.
### Migrate data from MySQL to PostgreSQL
Now, you can use pgloader to migrate the data from MySQL to PostgreSQL:
1. Save the following snippet in a `commands.load` file, and edit with your
MySQL `username`, `password` and `host`:
```
LOAD DATABASE
FROM mysql://username:password@host/gitlabhq_production
INTO postgresql://postgres@unix://var/run/postgresql:/gitlabhq_production
WITH include no drop, truncate, disable triggers, create no tables,
create no indexes, preserve index names, no foreign keys,
data only
ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public'
;
```
1. Start the migration:
``` bash
sudo -u postgres pgloader commands.load
```
1. Once the migration finishes, you should see a summary table that looks like
the following:
```
table name read imported errors total time
----------------------------------------------- --------- --------- --------- --------------
fetch meta data 119 119 0 0.388s
Truncate 119 119 0 1.134s
----------------------------------------------- --------- --------- --------- --------------
public.abuse_reports 0 0 0 0.490s
public.appearances 0 0 0 0.488s
.
.
.
public.web_hook_logs 0 0 0 1.080s
----------------------------------------------- --------- --------- --------- --------------
COPY Threads Completion 4 4 0 2.008s
Reset Sequences 113 113 0 0.304s
Install Comments 0 0 0 0.000s
----------------------------------------------- --------- --------- --------- --------------
Total import time 1894 1894 0 12.497s
```
If there is no output for more than 30 minutes, it's possible pgloader encountered an error. See
the [troubleshooting guide](#Troubleshooting) for more details.
1. Start GitLab:
``` bash
sudo service gitlab start
```
Now, you can verify that everything worked by visiting GitLab.
### Troubleshooting
#### Experiencing 500 errors after the migration
If you experience 500 errors after the migration, try to clear the cache:
``` bash
sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production
```
View git blame Copy permalink