Merge branch '52442-minimal-remove-mysql-support' into 'master'
Only support postgresql (minimal version) See merge request gitlab-org/gitlab-ce!29790
This commit is contained in:
Robert Speicher
commit
e66faa63ea
20 changed files with 29 additions and 530 deletions
|
|
@ -1,7 +1,6 @@
|
|||
image: "dev.gitlab.org:5005/gitlab/gitlab-build-images:ruby-2.6.3-golang-1.11-git-2.21-chrome-73.0-node-12.x-yarn-1.16-postgresql-9.6-graphicsmagick-1.3.29"
|
||||
|
||||
variables:
|
||||
MYSQL_ALLOW_EMPTY_PASSWORD: "1"
|
||||
RAILS_ENV: "test"
|
||||
NODE_ENV: "test"
|
||||
SIMPLECOV: "true"
|
||||
|
|
|
|||
|
|
@ -10,11 +10,6 @@
|
|||
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
|
||||
- name: redis:alpine
|
||||
|
||||
.use-mysql: &use-mysql
|
||||
services:
|
||||
- mysql:5.7
|
||||
- redis:alpine
|
||||
|
||||
.only-schedules-master: &only-schedules-master
|
||||
only:
|
||||
- schedules@gitlab-org/gitlab-ce
|
||||
|
|
@ -94,10 +89,6 @@
|
|||
<<: *use-pg-10
|
||||
image: "dev.gitlab.org:5005/gitlab/gitlab-build-images:ruby-2.6.3-golang-1.11-git-2.21-chrome-73.0-node-12.x-yarn-1.16-postgresql-10-graphicsmagick-1.3.29"
|
||||
|
||||
.rspec-metadata-mysql: &rspec-metadata-mysql
|
||||
<<: *rspec-metadata
|
||||
<<: *use-mysql
|
||||
|
||||
# DB migration, rollback, and seed jobs
|
||||
.db-migrate-reset: &db-migrate-reset
|
||||
extends: .dedicated-no-docs-and-no-qa-pull-cache-job
|
||||
|
|
@ -173,42 +164,6 @@ rspec system pg-10:
|
|||
<<: *only-schedules-master
|
||||
parallel: 24
|
||||
|
||||
rspec unit mysql:
|
||||
<<: *rspec-metadata-mysql
|
||||
<<: *only-schedules-master
|
||||
parallel: 20
|
||||
|
||||
rspec integration mysql:
|
||||
<<: *rspec-metadata-mysql
|
||||
<<: *only-schedules-master
|
||||
parallel: 6
|
||||
|
||||
rspec system mysql:
|
||||
<<: *rspec-metadata-mysql
|
||||
<<: *only-schedules-master
|
||||
parallel: 24
|
||||
|
||||
.rspec-mysql-on-demand: &rspec-mysql-on-demand
|
||||
only:
|
||||
variables:
|
||||
- $CI_COMMIT_MESSAGE =~ /\[run mysql\]/i
|
||||
- $CI_COMMIT_REF_NAME =~ /mysql/
|
||||
|
||||
rspec unit mysql on-demand:
|
||||
<<: *rspec-metadata-mysql
|
||||
<<: *rspec-mysql-on-demand
|
||||
parallel: 20
|
||||
|
||||
rspec integration mysql on-demand:
|
||||
<<: *rspec-metadata-mysql
|
||||
<<: *rspec-mysql-on-demand
|
||||
parallel: 6
|
||||
|
||||
rspec system mysql on-demand:
|
||||
<<: *rspec-metadata-mysql
|
||||
<<: *rspec-mysql-on-demand
|
||||
parallel: 24
|
||||
|
||||
rspec-fast-spec-helper:
|
||||
<<: *rspec-metadata-pg
|
||||
script:
|
||||
|
|
@ -226,12 +181,6 @@ rspec quarantine pg:
|
|||
<<: *rspec-quarantine
|
||||
allow_failure: true
|
||||
|
||||
rspec quarantine mysql:
|
||||
<<: *rspec-metadata-mysql
|
||||
<<: *rspec-quarantine
|
||||
<<: *only-schedules-master
|
||||
allow_failure: true
|
||||
|
||||
static-analysis:
|
||||
extends: .dedicated-no-docs-no-db-pull-cache-job
|
||||
dependencies:
|
||||
|
|
@ -281,10 +230,6 @@ db:migrate:reset-pg:
|
|||
<<: *db-migrate-reset
|
||||
<<: *use-pg
|
||||
|
||||
db:migrate:reset-mysql:
|
||||
<<: *db-migrate-reset
|
||||
<<: *use-mysql
|
||||
|
||||
db:check-schema-pg:
|
||||
<<: *db-migrate-reset
|
||||
<<: *use-pg
|
||||
|
|
@ -295,10 +240,6 @@ migration:path-pg:
|
|||
<<: *migration-paths
|
||||
<<: *use-pg
|
||||
|
||||
migration:path-mysql:
|
||||
<<: *migration-paths
|
||||
<<: *use-mysql
|
||||
|
||||
.db-rollback: &db-rollback
|
||||
extends: .dedicated-no-docs-and-no-qa-pull-cache-job
|
||||
script:
|
||||
|
|
@ -311,22 +252,12 @@ db:rollback-pg:
|
|||
<<: *db-rollback
|
||||
<<: *use-pg
|
||||
|
||||
db:rollback-mysql:
|
||||
<<: *db-rollback
|
||||
<<: *use-mysql
|
||||
|
||||
gitlab:setup-pg:
|
||||
<<: *gitlab-setup
|
||||
<<: *use-pg
|
||||
dependencies:
|
||||
- setup-test-env
|
||||
|
||||
gitlab:setup-mysql:
|
||||
<<: *gitlab-setup
|
||||
<<: *use-mysql
|
||||
dependencies:
|
||||
- setup-test-env
|
||||
|
||||
coverage:
|
||||
# Don't include dedicated-no-docs-no-db-pull-cache-job here since we need to
|
||||
# download artifacts from all the rspec jobs instead of from setup-test-env only
|
||||
|
|
|
|||
|
|
@ -0,0 +1,5 @@
|
|||
---
|
||||
title: Remove MySQL support
|
||||
merge_request: 29790
|
||||
author:
|
||||
type: removed
|
||||
4
config/initializers/1_postgresql_only.rb
Normal file
4
config/initializers/1_postgresql_only.rb
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
# frozen_string_literal: true
|
||||
|
||||
raise "PostgreSQL is the only supported database from GitLab 12.1" unless
|
||||
Gitlab::Database.postgresql?
|
||||
|
|
@ -42,7 +42,7 @@ The list of `GITLAB_DATABASE_XXX` variables that you can set is:
|
|||
|
||||
Variable | Default value | Overridden by `DATABASE_URL`?
|
||||
-------- | ------------- | -----------------------------
|
||||
`GITLAB_DATABASE_ADAPTER` | `postgresql` (for MySQL use `mysql2`) | Yes
|
||||
`GITLAB_DATABASE_ADAPTER` | `postgresql` | Yes
|
||||
`GITLAB_DATABASE_DATABASE` | `gitlab_#{ENV['RAILS_ENV']` | Yes
|
||||
`GITLAB_DATABASE_USERNAME` | `root` | Yes
|
||||
`GITLAB_DATABASE_PASSWORD` | None | Yes
|
||||
|
|
|
|||
|
|
@ -489,10 +489,6 @@ work:
|
|||
gitlab-ctl reconfigure
|
||||
```
|
||||
|
||||
## MySQL replication
|
||||
|
||||
MySQL replication is not supported for Geo.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
Read the [troubleshooting document](troubleshooting.md).
|
||||
|
|
|
|||
|
|
@ -33,7 +33,6 @@ Learn how to install, configure, update, and maintain your GitLab instance.
|
|||
|
||||
- [Install](../install/README.md): Requirements, directory structures, and installation methods.
|
||||
- [Database load balancing](database_load_balancing.md): Distribute database queries among multiple database servers. **[STARTER ONLY]**
|
||||
- [Omnibus support for external MySQL DB](https://docs.gitlab.com/omnibus/settings/database.html#using-a-mysql-database-management-server-enterprise-edition-only): Omnibus package supports configuring an external MySQL database. **[STARTER ONLY]**
|
||||
- [Omnibus support for log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only) **[STARTER ONLY]**
|
||||
- [High Availability](high_availability/README.md): Configure multiple servers for scaling or high availability.
|
||||
- [Installing GitLab HA on Amazon Web Services (AWS)](../install/aws/index.md): Set up GitLab High Availability on Amazon AWS.
|
||||
|
|
|
|||
|
|
@ -695,7 +695,7 @@ We've also detailed [our architecture of GitLab.com](https://about.gitlab.com/ha
|
|||
[runner-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/runner.md
|
||||
[database-migrations-omnibus]: https://docs.gitlab.com/omnibus/settings/database.html#disabling-automatic-database-migration
|
||||
[database-migrations-charts]: https://docs.gitlab.com/charts/charts/gitlab/migrations/
|
||||
[database-migrations-source]: ../update/upgrading_from_source.md#14-install-libs-migrations-etc
|
||||
[database-migrations-source]: ../update/upgrading_from_source.md#13-install-libs-migrations-etc
|
||||
[certificate-management-omnibus]: https://docs.gitlab.com/omnibus/settings/ssl.html
|
||||
[certificate-management-charts]: https://docs.gitlab.com/charts/installation/tls.html
|
||||
[certificate-management-source]: ../install/installation.md#using-https
|
||||
|
|
|
|||
|
|
@ -80,30 +80,6 @@ There are a few environment flags you can pass to change how projects are seeded
|
|||
- `LARGE_PROJECTS`: defaults to false. If set will clone 6 large projects to help with testing.
|
||||
- `FORK`: defaults to false. If set to `true` will fork `torvalds/linux` five times. Can also be set to an existing project full_path and it will fork that instead.
|
||||
|
||||
### Notes for MySQL
|
||||
|
||||
Since the seeds would contain various UTF-8 characters, such as emojis or so,
|
||||
we'll need to make sure that we're using `utf8mb4` for all the encoding
|
||||
settings and `utf8mb4_unicode_ci` for collation. Please check
|
||||
[MySQL utf8mb4 support](../install/database_mysql.md#mysql-utf8mb4-support)
|
||||
|
||||
Make sure that `config/database.yml` has `encoding: utf8mb4`, too.
|
||||
|
||||
Next, we'll need to update the schema to make the indices fit:
|
||||
|
||||
``` shell
|
||||
sed -i 's/limit: 255/limit: 191/g' db/schema.rb
|
||||
```
|
||||
|
||||
Then run the setup script:
|
||||
|
||||
``` shell
|
||||
bundle exec rake setup
|
||||
```
|
||||
|
||||
To make sure that indices still fit. You could find great details in:
|
||||
[How to support full Unicode in MySQL databases](https://mathiasbynens.be/notes/mysql-utf8mb4)
|
||||
|
||||
## Run tests
|
||||
|
||||
In order to run the test you can use the following commands:
|
||||
|
|
|
|||
|
|
@ -39,14 +39,6 @@ PostgreSQL you can use the `reset_pk_sequence!` method like so:
|
|||
reset_pk_sequence!('events')
|
||||
```
|
||||
|
||||
For MySQL however you need to do run the following:
|
||||
|
||||
```ruby
|
||||
amount = Event.pluck('COALESCE(MAX(id), 1)').first
|
||||
|
||||
execute "ALTER TABLE events AUTO_INCREMENT = #{amount}"
|
||||
```
|
||||
|
||||
Failure to reset the primary keys will result in newly created rows starting
|
||||
with an ID value of 1. Depending on the existing data this can then lead to
|
||||
duplicate key constraints from popping up, preventing users from creating new
|
||||
|
|
|
|||
|
|
@ -1,319 +0,0 @@
|
|||
---
|
||||
type: reference
|
||||
---
|
||||
|
||||
# Database MySQL
|
||||
|
||||
NOTE: **Note:**
|
||||
We do not recommend using MySQL due to various issues.
|
||||
For example, there have been bugs with case
|
||||
[(in)sensitivity](https://dev.mysql.com/doc/refman/5.7/en/case-sensitivity.html).
|
||||
|
||||
Bugs relating to case sensitivity:
|
||||
|
||||
- <https://bugs.mysql.com/bug.php?id=65830>
|
||||
- <https://bugs.mysql.com/bug.php?id=50909>
|
||||
- <https://bugs.mysql.com/bug.php?id=65830>
|
||||
- <https://bugs.mysql.com/bug.php?id=63164>
|
||||
|
||||
## Initial database setup
|
||||
|
||||
```sh
|
||||
# Install the database packages
|
||||
sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev
|
||||
|
||||
# Ensure you have MySQL version 5.6 or later
|
||||
mysql --version
|
||||
|
||||
# Pick a MySQL root password (can be anything), type it and press enter
|
||||
# Retype the MySQL root password and press enter
|
||||
|
||||
# Secure your installation
|
||||
sudo mysql_secure_installation
|
||||
|
||||
# Login to MySQL
|
||||
mysql -u root -p
|
||||
|
||||
# Type the MySQL root password
|
||||
|
||||
# Create a user for GitLab
|
||||
# do not type the 'mysql>', this is part of the prompt
|
||||
# change $password in the command below to a real password you pick
|
||||
mysql> CREATE USER 'git'@'localhost' IDENTIFIED BY '$password';
|
||||
|
||||
# Ensure you can use the InnoDB engine which is necessary to support long indexes
|
||||
# If this fails, check your MySQL config files (e.g. `/etc/mysql/*.cnf`, `/etc/mysql/conf.d/*`) for the setting "innodb = off"
|
||||
mysql> SET storage_engine=INNODB;
|
||||
|
||||
# If you have MySQL < 5.7.7 and want to enable utf8mb4 character set support with your GitLab install, you must set the following NOW:
|
||||
mysql> SET GLOBAL innodb_file_per_table=1, innodb_file_format=Barracuda, innodb_large_prefix=1;
|
||||
|
||||
# If you use MySQL with replication, or just have MySQL configured with binary logging, you need to run the following to allow the use of `TRIGGER`:
|
||||
mysql> SET GLOBAL log_bin_trust_function_creators = 1;
|
||||
|
||||
# Create the GitLab production database
|
||||
mysql> CREATE DATABASE IF NOT EXISTS `gitlabhq_production` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_general_ci`;
|
||||
|
||||
# Grant the GitLab user necessary permissions on the database
|
||||
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES, REFERENCES, TRIGGER ON `gitlabhq_production`.* TO 'git'@'localhost';
|
||||
|
||||
# Quit the database session
|
||||
mysql> \q
|
||||
|
||||
# Try connecting to the new database with the new user
|
||||
sudo -u git -H mysql -u git -p -D gitlabhq_production
|
||||
|
||||
# Type the password you replaced $password with earlier
|
||||
|
||||
# You should now see a 'mysql>' prompt
|
||||
|
||||
# Quit the database session
|
||||
mysql> \q
|
||||
```
|
||||
|
||||
You are done installing the database for now and can go back to the rest of the installation.
|
||||
Please proceed to the rest of the installation **before** running through the steps below.
|
||||
|
||||
### `log_bin_trust_function_creators`
|
||||
|
||||
If you use MySQL with replication, or just have MySQL configured with binary logging, all of your MySQL servers will need to have `log_bin_trust_function_creators` enabled to allow the use of `TRIGGER` in migrations. You have already set this global variable in the steps above, but to make it persistent, add the following to your `my.cnf` file:
|
||||
|
||||
```
|
||||
log_bin_trust_function_creators=1
|
||||
```
|
||||
|
||||
### MySQL utf8mb4 support
|
||||
|
||||
After installation or upgrade, remember to [convert any new tables](#tables-and-data-conversion-to-utf8mb4) to `utf8mb4`/`utf8mb4_general_ci`.
|
||||
|
||||
---
|
||||
|
||||
GitLab 8.14 has introduced [a feature](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7420) requiring `utf8mb4` encoding to be supported in your GitLab MySQL Database, which is not the case if you have set up your database before GitLab 8.16.
|
||||
|
||||
Follow the below instructions to ensure you use the most up to date requirements for your GitLab MySQL Database.
|
||||
|
||||
**We are about to do the following:**
|
||||
|
||||
- Ensure you can enable `utf8mb4` encoding and `utf8mb4_general_ci` collation for your GitLab DB, tables and data.
|
||||
- Convert your GitLab tables and data from `utf8`/`utf8_general_ci` to `utf8mb4`/`utf8mb4_general_ci`.
|
||||
|
||||
### Check for utf8mb4 support
|
||||
|
||||
#### Check for InnoDB File-Per-Table Tablespaces
|
||||
|
||||
We need to check, enable and maybe convert your existing GitLab DB tables to the [InnoDB File-Per-Table Tablespaces](https://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html) as a prerequisite for supporting **utfb8mb4 with long indexes** required by recent GitLab databases.
|
||||
|
||||
# Login to MySQL
|
||||
mysql -u root -p
|
||||
|
||||
# Type the MySQL root password
|
||||
mysql > use gitlabhq_production;
|
||||
|
||||
# Check your MySQL version is >= 5.5.3 (GitLab requires 5.5.14+)
|
||||
mysql > SHOW VARIABLES LIKE 'version';
|
||||
+---------------+-----------------+
|
||||
| Variable_name | Value |
|
||||
+---------------+-----------------+
|
||||
| version | 5.5.53-0+deb8u1 |
|
||||
+---------------+-----------------+
|
||||
|
||||
# Note where is your MySQL data dir for later:
|
||||
mysql > select @@datadir;
|
||||
+----------------+
|
||||
| @@datadir |
|
||||
+----------------+
|
||||
| /var/lib/mysql |
|
||||
+----------------+
|
||||
|
||||
# Note whether your MySQL server runs with innodb_file_per_table ON or OFF:
|
||||
mysql> SELECT @@innodb_file_per_table;
|
||||
+-------------------------+
|
||||
| @@innodb_file_per_table |
|
||||
+-------------------------+
|
||||
| 1 |
|
||||
+-------------------------+
|
||||
|
||||
# You can now quit the database session
|
||||
mysql> \q
|
||||
|
||||
> You need **MySQL 5.5.3 or later** to perform this update.
|
||||
|
||||
Whatever the results of your checks above, we now need to check if your GitLab database has been created using [InnoDB File-Per-Table Tablespaces](https://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html) (i.e. `innodb_file_per_table` was set to **1** at initial setup time).
|
||||
|
||||
NOTE: **Note:**
|
||||
This setting is [enabled by default](http://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_file_per_table) since MySQL 5.6.6.
|
||||
|
||||
# Run this command with root privileges, replace the data dir if different:
|
||||
sudo ls -lh /var/lib/mysql/gitlabhq_production/*.ibd | wc -l
|
||||
|
||||
# Run this command with root privileges, replace the data dir if different:
|
||||
sudo ls -lh /var/lib/mysql/gitlabhq_production/*.frm | wc -l
|
||||
|
||||
- **Case 1: a result > 0 for both commands**
|
||||
|
||||
Congratulations, your GitLab database uses the right InnoDB tablespace format.
|
||||
|
||||
However, you must still ensure that any **future tables** created by GitLab will still use the right format:
|
||||
|
||||
- If `SELECT @@innodb_file_per_table` returned **1** previously, your server is running correctly.
|
||||
|
||||
> It's however a requirement to check *now* that this setting is indeed persisted in your [`my.cnf`](https://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html) file!
|
||||
|
||||
- If `SELECT @@innodb_file_per_table` returned **0** previously, your server is not running correctly.
|
||||
|
||||
> [Enable innodb_file_per_table](https://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html) by running in a MySQL session as root the command `SET GLOBAL innodb_file_per_table=1, innodb_file_format=Barracuda;` and persist the two settings in your [`my.cnf`](https://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html) file.
|
||||
|
||||
Now, if you have a **different result** returned by the 2 commands above, it means you have a **mix of tables format** uses in your GitLab database. This can happen if your MySQL server had different values for `innodb_file_per_table` in its life and you updated GitLab at different moments with those inconsistent values. So keep reading.
|
||||
|
||||
- **Case 2: a result equals to "0" OR not the same result for both commands**
|
||||
|
||||
Unfortunately, none or only some of your GitLab database tables use the GitLab requirement of [InnoDB File-Per-Table Tablespaces](https://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html).
|
||||
|
||||
Let's enable what we need on the running server:
|
||||
|
||||
# Login to MySQL
|
||||
mysql -u root -p
|
||||
|
||||
# Type the MySQL root password
|
||||
|
||||
# Enable innodb_file_per_table and set innodb_file_format on the running server:
|
||||
mysql > SET GLOBAL innodb_file_per_table=1, innodb_file_format=Barracuda;
|
||||
|
||||
# You can now quit the database session
|
||||
mysql> \q
|
||||
|
||||
> Now, **persist** [innodb_file_per_table](https://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html) and [innodb_file_format](https://dev.mysql.com/doc/refman/5.7/en/innodb-file-format-enabling.html) in your `my.cnf` file.
|
||||
|
||||
Ensure at this stage that your GitLab instance is indeed **stopped**.
|
||||
|
||||
Now, let's convert all the GitLab database tables to the new tablespace format:
|
||||
|
||||
# Login to MySQL
|
||||
mysql -u root -p
|
||||
|
||||
# Type the MySQL root password
|
||||
mysql > use gitlabhq_production;
|
||||
|
||||
# Safety check: you should still have those values set as follows:
|
||||
mysql> SELECT @@innodb_file_per_table, @@innodb_file_format;
|
||||
+-------------------------+----------------------+
|
||||
| @@innodb_file_per_table | @@innodb_file_format |
|
||||
+-------------------------+----------------------+
|
||||
| 1 | Barracuda |
|
||||
+-------------------------+----------------------+
|
||||
|
||||
mysql > SELECT CONCAT('ALTER TABLE `', TABLE_NAME,'` ENGINE=InnoDB;') AS 'Copy & run these SQL statements:' FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA="gitlabhq_production" AND TABLE_TYPE="BASE TABLE";
|
||||
|
||||
# If previous query returned results, copy & run all shown SQL statements
|
||||
|
||||
# You can now quit the database session
|
||||
mysql> \q
|
||||
|
||||
---
|
||||
|
||||
#### Check for proper InnoDB File Format, Row Format, Large Prefix and tables conversion
|
||||
|
||||
We need to check, enable and probably convert your existing GitLab DB tables to use the [Barracuda InnoDB file format](https://dev.mysql.com/doc/refman/5.7/en/innodb-file-format.html), the [DYNAMIC row format](https://dev.mysql.com/doc/refman/5.7/en/glossary.html#glos_dynamic_row_format) and [innodb_large_prefix](http://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_large_prefix) as a second prerequisite for supporting **utfb8mb4 with long indexes** used by recent GitLab databases.
|
||||
|
||||
# Login to MySQL
|
||||
mysql -u root -p
|
||||
|
||||
# Type the MySQL root password
|
||||
mysql > use gitlabhq_production;
|
||||
|
||||
# Set innodb_file_format and innodb_large_prefix on the running server:
|
||||
# Note: These are the default settings only for MySQL 5.7.7 and later.
|
||||
|
||||
mysql > SET GLOBAL innodb_file_format=Barracuda, innodb_large_prefix=1;
|
||||
|
||||
# Your DB must be (still) using utf8/utf8_general_ci as default encoding and collation.
|
||||
# We will NOT change the default encoding and collation on the DB in order to support future GitLab migrations creating tables
|
||||
# that require "long indexes support" on installations using MySQL <= 5.7.9.
|
||||
# However, when such migrations occur, you will have to follow this guide again to convert the newly created tables to the proper encoding/collation.
|
||||
|
||||
# This should return the following:
|
||||
mysql> SELECT @@character_set_database, @@collation_database;
|
||||
+--------------------------+----------------------+
|
||||
| @@character_set_database | @@collation_database |
|
||||
+--------------------------+----------------------+
|
||||
| utf8 | utf8_general_ci |
|
||||
+--------------------------+----------------------+
|
||||
|
||||
> Now, ensure that [innodb_file_format](https://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html) and [innodb_large_prefix](http://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_large_prefix) are **persisted** in your `my.cnf` file.
|
||||
|
||||
#### Tables and data conversion to utf8mb4
|
||||
|
||||
Now that you have a persistent MySQL setup, you can safely upgrade tables after setup or upgrade time:
|
||||
|
||||
# Convert tables not using ROW_FORMAT DYNAMIC:
|
||||
|
||||
mysql> SELECT CONCAT('ALTER TABLE `', TABLE_NAME,'` ROW_FORMAT=DYNAMIC;') AS 'Copy & run these SQL statements:' FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA="gitlabhq_production" AND TABLE_TYPE="BASE TABLE" AND ROW_FORMAT!="Dynamic";
|
||||
|
||||
# !! If previous query returned results, copy & run all shown SQL statements
|
||||
|
||||
# Convert tables/columns not using utf8mb4/utf8mb4_general_ci as encoding/collation:
|
||||
|
||||
mysql > SET foreign_key_checks = 0;
|
||||
|
||||
mysql > SELECT CONCAT('ALTER TABLE `', TABLE_NAME,'` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;') AS 'Copy & run these SQL statements:' FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA="gitlabhq_production" AND TABLE_COLLATION != "utf8mb4_general_ci" AND TABLE_TYPE="BASE TABLE";
|
||||
|
||||
# !! If previous query returned results, copy & run all shown SQL statements
|
||||
|
||||
# Turn foreign key checks back on
|
||||
mysql > SET foreign_key_checks = 1;
|
||||
|
||||
# You can now quit the database session
|
||||
mysql> \q
|
||||
|
||||
Ensure your GitLab database configuration file uses a proper connection encoding and collation:
|
||||
|
||||
`sudo -u git -H editor config/database.yml`
|
||||
|
||||
production:
|
||||
adapter: mysql2
|
||||
encoding: utf8mb4
|
||||
collation: utf8mb4_general_ci
|
||||
|
||||
[Restart your GitLab instance](../administration/restart_gitlab.md).
|
||||
|
||||
## MySQL strings limits
|
||||
|
||||
After installation or upgrade, remember to run the `add_limits_mysql` Rake task:
|
||||
|
||||
**Omnibus GitLab installations**
|
||||
|
||||
```sh
|
||||
sudo gitlab-rake add_limits_mysql
|
||||
```
|
||||
|
||||
**Installations from source**
|
||||
|
||||
```sh
|
||||
bundle exec rake add_limits_mysql RAILS_ENV=production
|
||||
```
|
||||
|
||||
The `text` type in MySQL has a different size limit than the `text` type in
|
||||
PostgreSQL. In MySQL `text` columns are limited to ~65kB, whereas in PostgreSQL
|
||||
`text` columns are limited up to ~1GB!
|
||||
|
||||
The `add_limits_mysql` Rake task converts some important `text` columns in the
|
||||
GitLab database to `longtext` columns, which can persist values of up to 4GB
|
||||
(sometimes less if the value contains multibyte characters).
|
||||
|
||||
Details can be found in the [PostgreSQL][postgres-text-type] and
|
||||
[MySQL](https://dev.mysql.com/doc/refman/5.7/en/string-type-overview.html) manuals.
|
||||
|
||||
[postgres-text-type]: http://www.postgresql.org/docs/9.2/static/datatype-character.html
|
||||
[ce-38152]: https://gitlab.com/gitlab-org/gitlab-ce/issues/38152
|
||||
|
||||
<!-- ## Troubleshooting
|
||||
|
||||
Include any troubleshooting steps that you can foresee. If you know beforehand what issues
|
||||
one might have when setting this up, or when something is changed, or on upgrading, it's
|
||||
important to describe those, too. Think of things that may go wrong and include them here.
|
||||
This is important to minimize requests for support and to avoid doc comments with
|
||||
questions that you know someone might ask.
|
||||
|
||||
Each scenario can be a third-level heading, e.g. `### Getting error message X`.
|
||||
If you have none to add when creating a doc, leave this section in place
|
||||
but commented out to help encourage others to add to it in the future. -->
|
||||
|
|
@ -293,10 +293,9 @@ sudo adduser --disabled-login --gecos 'GitLab' git
|
|||
|
||||
## 6. Database
|
||||
|
||||
We recommend using a PostgreSQL database. For MySQL, see the [MySQL setup guide](database_mysql.md).
|
||||
|
||||
NOTE: **Note:**
|
||||
Because we need to make use of extensions and concurrent index removal, you need at least PostgreSQL 9.2.
|
||||
Starting from GitLab 12.1, only PostgreSQL is supported. Because we need to make
|
||||
use of extensions and concurrent index removal, you need at least PostgreSQL 9.2.
|
||||
|
||||
1. Install the database packages:
|
||||
|
||||
|
|
@ -502,13 +501,8 @@ If you want to use HTTPS, see [Using HTTPS](#using-https) for the additional ste
|
|||
### Configure GitLab DB Settings
|
||||
|
||||
```sh
|
||||
# PostgreSQL only:
|
||||
sudo -u git cp config/database.yml.postgresql config/database.yml
|
||||
|
||||
# MySQL only:
|
||||
sudo -u git cp config/database.yml.mysql config/database.yml
|
||||
|
||||
# PostgreSQL only:
|
||||
# Remove host, username, and password lines from config/database.yml.
|
||||
# Once modified, the `production` settings will be as follows:
|
||||
#
|
||||
|
|
@ -520,7 +514,7 @@ sudo -u git cp config/database.yml.mysql config/database.yml
|
|||
#
|
||||
sudo -u git -H editor config/database.yml
|
||||
|
||||
# MySQL and remote PostgreSQL only:
|
||||
# Remote PostgreSQL only:
|
||||
# Update username/password in config/database.yml.
|
||||
# You only need to adapt the production settings (first part).
|
||||
# If you followed the database guide then please do as follows:
|
||||
|
|
@ -528,7 +522,6 @@ sudo -u git -H editor config/database.yml
|
|||
# You can keep the double quotes around the password
|
||||
sudo -u git -H editor config/database.yml
|
||||
|
||||
# PostgreSQL and MySQL:
|
||||
# Make config/database.yml readable to git only
|
||||
sudo -u git -H chmod o-rwx config/database.yml
|
||||
```
|
||||
|
|
@ -544,11 +537,7 @@ Make sure you have `bundle` (run `bundle -v`):
|
|||
- `< 2.x`.
|
||||
|
||||
```sh
|
||||
# For PostgreSQL (note, the option says "without ... mysql")
|
||||
sudo -u git -H bundle install --deployment --without development test mysql aws kerberos
|
||||
|
||||
# Or if you use MySQL (note, the option says "without ... postgres")
|
||||
sudo -u git -H bundle install --deployment --without development test postgres aws kerberos
|
||||
```
|
||||
|
||||
NOTE: **Note:**
|
||||
|
|
|
|||
|
|
@ -104,32 +104,10 @@ installation (e.g. the number of users, projects, etc).
|
|||
|
||||
We currently support the following databases:
|
||||
|
||||
- PostgreSQL (highly recommended)
|
||||
- MySQL/MariaDB (strongly discouraged, not all GitLab features are supported, no support for [MySQL/MariaDB GTID](https://mariadb.com/kb/en/mariadb/gtid/))
|
||||
- PostgreSQL
|
||||
|
||||
We highly recommend the use of PostgreSQL instead of MySQL/MariaDB as not all
|
||||
features of GitLab work with MySQL/MariaDB:
|
||||
|
||||
1. MySQL support for subgroups was [dropped with GitLab 9.3][post].
|
||||
See [issue #30472][30472] for more information.
|
||||
1. Geo does [not support MySQL](../administration/geo/replication/database.md). This means no supported Disaster Recovery solution if using MySQL. **[PREMIUM ONLY]**
|
||||
1. [Zero downtime migrations](../update/README.md#upgrading-without-downtime) do not work with MySQL.
|
||||
1. [Database load balancing](../administration/database_load_balancing.md) is
|
||||
supported only for PostgreSQL. **[PREMIUM ONLY]**
|
||||
1. GitLab [optimizes the loading of dashboard events](https://gitlab.com/gitlab-org/gitlab-ce/issues/31806) using [PostgreSQL LATERAL JOINs](https://blog.heapanalytics.com/postgresqls-powerful-new-join-type-lateral/).
|
||||
1. In general, SQL optimized for PostgreSQL may run much slower in MySQL due to
|
||||
differences in query planners. For example, subqueries that work well in PostgreSQL
|
||||
may not be [performant in MySQL](https://dev.mysql.com/doc/refman/5.7/en/optimizing-subqueries.html).
|
||||
1. Binary column index length is limited to 20 bytes. This is accomplished with [a hack](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/initializers/mysql_set_length_for_binary_indexes.rb).
|
||||
1. MySQL requires a variety of hacks to increase limits on various columns, [for example](https://gitlab.com/gitlab-org/gitlab-ce/issues/49583).
|
||||
1. [The milestone filter runs slower queries on MySQL](https://gitlab.com/gitlab-org/gitlab-ce/issues/51173#note_99391731).
|
||||
1. We expect this list to grow over time.
|
||||
|
||||
Existing users using GitLab with MySQL/MariaDB are advised to
|
||||
[migrate to PostgreSQL](../update/mysql_to_postgresql.md) instead.
|
||||
|
||||
[30472]: https://gitlab.com/gitlab-org/gitlab-ce/issues/30472
|
||||
[post]: https://about.gitlab.com/2017/06/22/gitlab-9-3-released/#dropping-support-for-subgroups-in-mysql
|
||||
Support for MySQL was removed in GitLab 12.1. Existing users using GitLab with
|
||||
MySQL/MariaDB are advised to [migrate to PostgreSQL](../update/mysql_to_postgresql.md) before upgrading.
|
||||
|
||||
### PostgreSQL Requirements
|
||||
|
||||
|
|
|
|||
|
|
@ -194,11 +194,7 @@ from the Omniauth provider's documentation.
|
|||
|
||||
gem "omniauth-your-auth-provider"
|
||||
|
||||
- If you're using MySQL, install the new Omniauth provider gem by running the following command:
|
||||
|
||||
sudo -u git -H bundle install --without development test postgres --path vendor/bundle --no-deployment
|
||||
|
||||
- If you're using PostgreSQL, install the new Omniauth provider gem by running the following command:
|
||||
- Install the new Omniauth provider gem by running the following command:
|
||||
|
||||
sudo -u git -H bundle install --without development test mysql --path vendor/bundle --no-deployment
|
||||
|
||||
|
|
|
|||
|
|
@ -149,7 +149,6 @@ The GitLab University curriculum is composed of GitLab videos, screencasts, pres
|
|||
1. [How to Install GitLab with Omnibus - Video](https://www.youtube.com/watch?v=Q69YaOjqNhg)
|
||||
1. [Installing GitLab - Online Course](https://courses.platzi.com/classes/git-gitlab/concepto/part-1/part-3/material/)
|
||||
1. [Using a Non-Packaged PostgreSQL Database](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#using-a-non-packaged-postgresql-database-management-server)
|
||||
1. [Using a MySQL Database](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#using-a-mysql-database-management-server-enterprise-edition-only)
|
||||
1. [Installing GitLab on Microsoft Azure](https://about.gitlab.com/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/)
|
||||
1. [Installing GitLab on Digital Ocean](https://about.gitlab.com/2016/04/27/getting-started-with-gitlab-and-digitalocean/)
|
||||
|
||||
|
|
|
|||
|
|
@ -49,8 +49,7 @@ However, for this to work there are the following requirements:
|
|||
- You have to use [post-deployment
|
||||
migrations](../development/post_deployment_migrations.md) (included in
|
||||
zero downtime update steps below).
|
||||
- You are using PostgreSQL. If you are using MySQL please look at the release
|
||||
post to see if downtime is required.
|
||||
- You are using PostgreSQL. Starting from GitLab 12.1, MySQL is not supported.
|
||||
|
||||
Most of the time you can safely upgrade from a patch release to the next minor
|
||||
release if the patch release is not the latest. For example, upgrading from
|
||||
|
|
@ -140,8 +139,6 @@ possible.
|
|||
|
||||
- [MySQL to PostgreSQL](mysql_to_postgresql.md) guides you through migrating
|
||||
your database from MySQL to PostgreSQL.
|
||||
- [MySQL installation guide](../install/database_mysql.md) contains additional
|
||||
information about configuring GitLab to work with a MySQL database.
|
||||
- [Restoring from backup after a failed upgrade](restore_after_failure.md)
|
||||
- [Upgrading PostgreSQL Using Slony](upgrading_postgresql_using_slony.md), for
|
||||
upgrading a PostgreSQL database with minimal downtime.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
last_updated: 2019-03-27
|
||||
last_updated: 2019-06-18
|
||||
---
|
||||
|
||||
# Migrating from MySQL to PostgreSQL
|
||||
|
|
@ -9,6 +9,10 @@ migrate it to a PostgreSQL database.
|
|||
|
||||
## Requirements
|
||||
|
||||
NOTE: **Note:**
|
||||
Support for MySQL was removed in GitLab 12.1. This procedure should be performed
|
||||
**before** installing GitLab 12.1.
|
||||
|
||||
[pgloader](http://pgloader.io) 3.4.1+ is required.
|
||||
|
||||
You can install it directly from your distribution, for example in
|
||||
|
|
|
|||
|
|
@ -13,8 +13,6 @@ You can select the tag in the version dropdown in the top left corner of GitLab
|
|||
### 0. Backup
|
||||
|
||||
It's useful to make a backup just in case things go south:
|
||||
(With MySQL, this may require granting "LOCK TABLES" privileges to the GitLab
|
||||
user on the database version)
|
||||
|
||||
```bash
|
||||
cd /home/git/gitlab
|
||||
|
|
@ -48,12 +46,8 @@ sudo -u git -H git checkout LATEST_TAG -b LATEST_TAG
|
|||
```bash
|
||||
cd /home/git/gitlab
|
||||
|
||||
# PostgreSQL
|
||||
sudo -u git -H bundle install --without development test mysql --deployment
|
||||
|
||||
# MySQL
|
||||
sudo -u git -H bundle install --without development test postgres --deployment
|
||||
|
||||
# Optional: clean up old gems
|
||||
sudo -u git -H bundle clean
|
||||
|
||||
|
|
|
|||
|
|
@ -239,29 +239,7 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_PAGES_VERSION)
|
|||
sudo -u git -H make
|
||||
```
|
||||
|
||||
### 12. Update MySQL permissions
|
||||
|
||||
If you are using MySQL you need to grant the GitLab user the necessary
|
||||
permissions on the database:
|
||||
|
||||
```bash
|
||||
mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';"
|
||||
```
|
||||
|
||||
If you use MySQL with replication, or just have MySQL configured with binary logging,
|
||||
you will need to also run the following on all of your MySQL servers:
|
||||
|
||||
```bash
|
||||
mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;"
|
||||
```
|
||||
|
||||
You can make this setting permanent by adding it to your `my.cnf`:
|
||||
|
||||
```
|
||||
log_bin_trust_function_creators=1
|
||||
```
|
||||
|
||||
### 13. Update configuration files
|
||||
### 12. Update configuration files
|
||||
|
||||
#### New configuration options for `gitlab.yml`
|
||||
|
||||
|
|
@ -335,18 +313,13 @@ For Ubuntu 16.04.1 LTS:
|
|||
sudo systemctl daemon-reload
|
||||
```
|
||||
|
||||
### 14. Install libs, migrations, etc.
|
||||
### 13. Install libs, migrations, etc.
|
||||
|
||||
```bash
|
||||
cd /home/git/gitlab
|
||||
|
||||
# PostgreSQL installations (note: the line below states '--without mysql')
|
||||
sudo -u git -H bundle install --deployment --without development test mysql aws kerberos
|
||||
|
||||
# MySQL installations (note: the line below states '--without postgres')
|
||||
sudo -u git -H bundle install --deployment --without development test postgres aws kerberos
|
||||
|
||||
|
||||
# Optional: clean up old gems
|
||||
sudo -u git -H bundle clean
|
||||
|
||||
|
|
@ -364,17 +337,14 @@ sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:c
|
|||
sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production
|
||||
```
|
||||
|
||||
**MySQL installations**: Run through the `MySQL strings limits` and `Tables and
|
||||
data conversion to utf8mb4` [tasks](../install/database_mysql.md).
|
||||
|
||||
### 15. Start application
|
||||
### 14. Start application
|
||||
|
||||
```bash
|
||||
sudo service gitlab start
|
||||
sudo service nginx restart
|
||||
```
|
||||
|
||||
### 16. Check application status
|
||||
### 15. Check application status
|
||||
|
||||
Check if GitLab and its environment are configured correctly:
|
||||
|
||||
|
|
|
|||
|
|
@ -5,8 +5,7 @@ type: reference, howto, concepts
|
|||
# Subgroups
|
||||
|
||||
NOTE: **Note:**
|
||||
[Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/2772) in GitLab 9.0. Not available when using MySQL as external
|
||||
database (support removed in GitLab 9.3 [due to performance reasons](https://gitlab.com/gitlab-org/gitlab-ce/issues/30472#note_27747600)).
|
||||
[Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/2772) in GitLab 9.0.
|
||||
|
||||
Subgroups, also known as nested groups or hierarchical groups, allow you to have up to 20
|
||||
levels of groups.
|
||||
|
|
@ -21,16 +20,6 @@ By using subgroups you can do the following:
|
|||
- **Make it easier to manage people and control visibility.** Give people
|
||||
different [permissions](../../permissions.md#group-members-permissions) depending on their group [membership](#membership).
|
||||
|
||||
## Database Requirements
|
||||
|
||||
Subgroups are only supported when you use PostgreSQL. Supporting subgroups on MySQL in an
|
||||
efficient way is not possible due to MySQL's limitations.
|
||||
|
||||
See the following links for more information:
|
||||
|
||||
- <https://gitlab.com/gitlab-org/gitlab-ce/issues/30472>
|
||||
- <https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10885>
|
||||
|
||||
## Overview
|
||||
|
||||
A group can have many subgroups inside it, and at the same time a group can have
|
||||
|
|
|
|||
Loading…
Reference in a new issue