Remove MySQL references from development docs
I noticed the doc/development/testing_guide/best_practices.md still referenced the `[run mysql]` tags, etc. They no longer work, so I removed them, then realised I had better clean up the rest of doc/development !
This commit is contained in:
parent
95075fee78
commit
5a574883f9
10 changed files with 34 additions and 72 deletions
|
@ -20,7 +20,7 @@ A typical install of GitLab will be on GNU/Linux. It uses Nginx or Apache as a w
|
|||
|
||||
We also support deploying GitLab on Kubernetes using our [gitlab Helm chart](https://docs.gitlab.com/charts/).
|
||||
|
||||
The GitLab web app uses MySQL or PostgreSQL for persistent database information (e.g. users, permissions, issues, other meta data). GitLab stores the bare git repositories it serves in `/home/git/repositories` by default. It also keeps default branch and hook information with the bare repository.
|
||||
The GitLab web app uses PostgreSQL for persistent database information (e.g. users, permissions, issues, other meta data). GitLab stores the bare git repositories it serves in `/home/git/repositories` by default. It also keeps default branch and hook information with the bare repository.
|
||||
|
||||
When serving repositories over HTTP/HTTPS GitLab utilizes the GitLab API to resolve authorization and access as well as serving git objects.
|
||||
|
||||
|
@ -511,7 +511,15 @@ To summarize here's the [directory structure of the `git` user home directory](.
|
|||
ps aux | grep '^git'
|
||||
```
|
||||
|
||||
GitLab has several components to operate. As a system user (i.e. any user that is not the `git` user) it requires a persistent database (MySQL/PostreSQL) and redis database. It also uses Apache httpd or Nginx to proxypass Unicorn. As the `git` user it starts Sidekiq and Unicorn (a simple ruby HTTP server running on port `8080` by default). Under the GitLab user there are normally 4 processes: `unicorn_rails master` (1 process), `unicorn_rails worker` (2 processes), `sidekiq` (1 process).
|
||||
GitLab has several components to operate. It requires a persistent database
|
||||
(PostgreSQL) and redis database, and uses Apache httpd or Nginx to proxypass
|
||||
Unicorn. All these components should run as different system users to GitLab
|
||||
(e.g., `postgres`, `redis` and `www-data`, instead of `git`).
|
||||
|
||||
As the `git` user it starts Sidekiq and Unicorn (a simple ruby HTTP server
|
||||
running on port `8080` by default). Under the GitLab user there are normally 4
|
||||
processes: `unicorn_rails master` (1 process), `unicorn_rails worker`
|
||||
(2 processes), `sidekiq` (1 process).
|
||||
|
||||
### Repository access
|
||||
|
||||
|
@ -554,12 +562,9 @@ $ /etc/init.d/nginx
|
|||
Usage: nginx {start|stop|restart|reload|force-reload|status|configtest}
|
||||
```
|
||||
|
||||
Persistent database (one of the following)
|
||||
Persistent database
|
||||
|
||||
```
|
||||
/etc/init.d/mysqld
|
||||
Usage: /etc/init.d/mysqld {start|stop|status|restart|condrestart|try-restart|reload|force-reload}
|
||||
|
||||
$ /etc/init.d/postgresql
|
||||
Usage: /etc/init.d/postgresql {start|stop|restart|reload|force-reload|status} [version ..]
|
||||
```
|
||||
|
@ -597,11 +602,6 @@ PostgreSQL
|
|||
|
||||
- `/var/log/postgresql/*`
|
||||
|
||||
MySQL
|
||||
|
||||
- `/var/log/mysql/*`
|
||||
- `/var/log/mysql.*`
|
||||
|
||||
### GitLab specific config files
|
||||
|
||||
GitLab has configuration files located in `/home/git/gitlab/config/*`. Commonly referenced config files include:
|
||||
|
|
|
@ -1,6 +1,6 @@
|
|||
# Hash Indexes
|
||||
|
||||
Both PostgreSQL and MySQL support hash indexes besides the regular btree
|
||||
PostgreSQL supports hash indexes besides the regular btree
|
||||
indexes. Hash indexes however are to be avoided at all costs. While they may
|
||||
_sometimes_ provide better performance the cost of rehashing can be very high.
|
||||
More importantly: at least until PostgreSQL 10.0 hash indexes are not
|
||||
|
|
|
@ -9,7 +9,7 @@ bundle exec rake setup
|
|||
```
|
||||
|
||||
The `setup` task is an alias for `gitlab:setup`.
|
||||
This tasks calls `db:reset` to create the database, calls `add_limits_mysql` that adds limits to the database schema in case of a MySQL database and finally it calls `db:seed_fu` to seed the database.
|
||||
This tasks calls `db:reset` to create the database, and calls `db:seed_fu` to seed the database.
|
||||
Note: `db:setup` calls `db:seed` but this does nothing.
|
||||
|
||||
### Seeding issues for all or a given project
|
||||
|
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Storing SHA1 hashes as strings is not very space efficient. A SHA1 as a string
|
||||
requires at least 40 bytes, an additional byte to store the encoding, and
|
||||
perhaps more space depending on the internals of PostgreSQL and MySQL.
|
||||
perhaps more space depending on the internals of PostgreSQL.
|
||||
|
||||
On the other hand, if one were to store a SHA1 as binary one would only need 20
|
||||
bytes for the actual SHA1, and 1 or 4 bytes of additional space (again depending
|
||||
|
|
|
@ -15,14 +15,11 @@ FROM issues
|
|||
WHERE title LIKE 'WIP:%';
|
||||
```
|
||||
|
||||
On PostgreSQL the `LIKE` statement is case-sensitive. On MySQL this depends on
|
||||
the case-sensitivity of the collation, which is usually case-insensitive. To
|
||||
perform a case-insensitive `LIKE` on PostgreSQL you have to use `ILIKE` instead.
|
||||
This statement in turn isn't supported on MySQL.
|
||||
On PostgreSQL the `LIKE` statement is case-sensitive. To perform a case-insensitive
|
||||
`LIKE` you have to use `ILIKE` instead.
|
||||
|
||||
To work around this problem you should write `LIKE` queries using Arel instead
|
||||
of raw SQL fragments as Arel automatically uses `ILIKE` on PostgreSQL and `LIKE`
|
||||
on MySQL. This means that instead of this:
|
||||
To handle this automatically you should use `LIKE` queries using Arel instead
|
||||
of raw SQL fragments, as Arel automatically uses `ILIKE` on PostgreSQL.
|
||||
|
||||
```ruby
|
||||
Issue.where('title LIKE ?', 'WIP:%')
|
||||
|
@ -45,7 +42,7 @@ table = Issue.arel_table
|
|||
Issue.where(table[:title].matches('WIP:%').or(table[:foo].matches('WIP:%')))
|
||||
```
|
||||
|
||||
For PostgreSQL this produces:
|
||||
On PostgreSQL, this produces:
|
||||
|
||||
```sql
|
||||
SELECT *
|
||||
|
@ -53,18 +50,10 @@ FROM issues
|
|||
WHERE (title ILIKE 'WIP:%' OR foo ILIKE 'WIP:%')
|
||||
```
|
||||
|
||||
In turn for MySQL this produces:
|
||||
|
||||
```sql
|
||||
SELECT *
|
||||
FROM issues
|
||||
WHERE (title LIKE 'WIP:%' OR foo LIKE 'WIP:%')
|
||||
```
|
||||
|
||||
## LIKE & Indexes
|
||||
|
||||
Neither PostgreSQL nor MySQL use any indexes when using `LIKE` / `ILIKE` with a
|
||||
wildcard at the start. For example, this will not use any indexes:
|
||||
PostgreSQL won't use any indexes when using `LIKE` / `ILIKE` with a wildcard at
|
||||
the start. For example, this will not use any indexes:
|
||||
|
||||
```sql
|
||||
SELECT *
|
||||
|
@ -75,9 +64,8 @@ WHERE title ILIKE '%WIP:%';
|
|||
Because the value for `ILIKE` starts with a wildcard the database is not able to
|
||||
use an index as it doesn't know where to start scanning the indexes.
|
||||
|
||||
MySQL provides no known solution to this problem. Luckily PostgreSQL _does_
|
||||
provide a solution: trigram GIN indexes. These indexes can be created as
|
||||
follows:
|
||||
Luckily, PostgreSQL _does_ provide a solution: trigram GIN indexes. These
|
||||
indexes can be created as follows:
|
||||
|
||||
```sql
|
||||
CREATE INDEX [CONCURRENTLY] index_name_here
|
||||
|
|
|
@ -15,16 +15,6 @@ manifest themselves within our code. When designing our tests, take time to revi
|
|||
our test design. We can find some helpful heuristics documented in the Handbook in the
|
||||
[Test Design](https://about.gitlab.com/handbook/engineering/quality/guidelines/test-engineering/test-design/) section.
|
||||
|
||||
## Run tests against MySQL
|
||||
|
||||
By default, tests are only run against PostgreSQL, but you can run them on
|
||||
demand against MySQL by following one of the following conventions:
|
||||
|
||||
| Convention | Valid example |
|
||||
|:----------------------|:-----------------------------|
|
||||
| Include `mysql` in your branch name | `enhance-mysql-support` |
|
||||
| Include `[run mysql]` in your commit message | `Fix MySQL support<br><br>[run mysql]` |
|
||||
|
||||
## Test speed
|
||||
|
||||
GitLab has a massive test suite that, without [parallelization], can take hours
|
||||
|
|
|
@ -39,7 +39,6 @@ slowest test files and try to improve them.
|
|||
|
||||
## CI setup
|
||||
|
||||
- On CE and EE, the test suite runs both PostgreSQL and MySQL.
|
||||
- Rails logging to `log/test.log` is disabled by default in CI [for
|
||||
performance reasons][logging]. To override this setting, provide the
|
||||
`RAILS_ENABLE_TEST_LOG` environment variable.
|
||||
|
|
|
@ -35,8 +35,8 @@ Once a test is in quarantine, there are 3 choices:
|
|||
|
||||
Quarantined tests are run on the CI in dedicated jobs that are allowed to fail:
|
||||
|
||||
- `rspec-pg-quarantine` and `rspec-mysql-quarantine` (CE & EE)
|
||||
- `rspec-pg-quarantine-ee` and `rspec-mysql-quarantine-ee` (EE only)
|
||||
- `rspec-pg-quarantine` (CE & EE)
|
||||
- `rspec-pg-quarantine-ee` (EE only)
|
||||
|
||||
## Automatic retries and flaky tests detection
|
||||
|
||||
|
|
|
@ -1,15 +1,15 @@
|
|||
# Verifying Database Capabilities
|
||||
|
||||
Sometimes certain bits of code may only work on a certain database and/or
|
||||
Sometimes certain bits of code may only work on a certain database
|
||||
version. While we try to avoid such code as much as possible sometimes it is
|
||||
necessary to add database (version) specific behaviour.
|
||||
|
||||
To facilitate this we have the following methods that you can use:
|
||||
|
||||
- `Gitlab::Database.postgresql?`: returns `true` if PostgreSQL is being used
|
||||
- `Gitlab::Database.mysql?`: returns `true` if MySQL is being used
|
||||
- `Gitlab::Database.postgresql?`: returns `true` if PostgreSQL is being used.
|
||||
You can normally just assume this is the case.
|
||||
- `Gitlab::Database.version`: returns the PostgreSQL version number as a string
|
||||
in the format `X.Y.Z`. This method does not work for MySQL
|
||||
in the format `X.Y.Z`.
|
||||
|
||||
This allows you to write code such as:
|
||||
|
||||
|
|
|
@ -7,9 +7,8 @@ downtime.
|
|||
|
||||
## Adding Columns
|
||||
|
||||
On PostgreSQL you can safely add a new column to an existing table as long as it
|
||||
does **not** have a default value. For example, this query would not require
|
||||
downtime:
|
||||
You can safely add a new column to an existing table as long as it does **not**
|
||||
have a default value. For example, this query would not require downtime:
|
||||
|
||||
```sql
|
||||
ALTER TABLE projects ADD COLUMN random_value int;
|
||||
|
@ -27,11 +26,6 @@ This requires updating every single row in the `projects` table so that
|
|||
indexes in a table. This in turn acquires enough locks on the table for it to
|
||||
effectively block any other queries.
|
||||
|
||||
As of MySQL 5.6 adding a column to a table is still quite an expensive
|
||||
operation, even when using `ALGORITHM=INPLACE` and `LOCK=NONE`. This means
|
||||
downtime _may_ be required when modifying large tables as otherwise the
|
||||
operation could potentially take hours to complete.
|
||||
|
||||
Adding a column with a default value _can_ be done without requiring downtime
|
||||
when using the migration helper method
|
||||
`Gitlab::Database::MigrationHelpers#add_column_with_default`. This method works
|
||||
|
@ -311,8 +305,7 @@ migrations](background_migrations.md#cleaning-up).
|
|||
## Adding Indexes
|
||||
|
||||
Adding indexes is an expensive process that blocks INSERT and UPDATE queries for
|
||||
the duration. When using PostgreSQL one can work around this by using the
|
||||
`CONCURRENTLY` option:
|
||||
the duration. You can work around this by using the `CONCURRENTLY` option:
|
||||
|
||||
```sql
|
||||
CREATE INDEX CONCURRENTLY index_name ON projects (column_name);
|
||||
|
@ -336,17 +329,9 @@ end
|
|||
Note that `add_concurrent_index` can not be reversed automatically, thus you
|
||||
need to manually define `up` and `down`.
|
||||
|
||||
When running this on PostgreSQL the `CONCURRENTLY` option mentioned above is
|
||||
used. On MySQL this method produces a regular `CREATE INDEX` query.
|
||||
|
||||
MySQL doesn't really have a workaround for this. Supposedly it _can_ create
|
||||
indexes without the need for downtime but only for variable width columns. The
|
||||
details on this are a bit sketchy. Since it's better to be safe than sorry one
|
||||
should assume that adding indexes requires downtime on MySQL.
|
||||
|
||||
## Dropping Indexes
|
||||
|
||||
Dropping an index does not require downtime on both PostgreSQL and MySQL.
|
||||
Dropping an index does not require downtime.
|
||||
|
||||
## Adding Tables
|
||||
|
||||
|
@ -370,7 +355,7 @@ transaction this means this approach would require downtime.
|
|||
|
||||
GitLab allows you to work around this by using
|
||||
`Gitlab::Database::MigrationHelpers#add_concurrent_foreign_key`. This method
|
||||
ensures that when PostgreSQL is used no downtime is needed.
|
||||
ensures that no downtime is needed.
|
||||
|
||||
## Removing Foreign Keys
|
||||
|
||||
|
|
Loading…
Reference in a new issue