kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Improve Hashed Storage documentation for rollback

Browse source 
Fixed storage coverage table with additional information
and wrote down implementationd details from few entities.
This commit is contained in:
Gabriel Mazetto 2018-06-27 05:01:09 +02:00
parent 869d645069
commit fbc687c032
1 changed files with 66 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

68
doc/administration/repository_storage_types.md
View file

@ -82,6 +82,46 @@ To migrate your existing projects to the new storage type, check the specific
[rake tasks]: raketasks/storage.md#migrate-existing-projects-to-hashed-storage
[storage-paths]: repository_storage_types.md
#### Rollback
There is no automated rollback implemented. Below are the steps required to rollback
from each storage migration.
The rollback has to be performed in the reverse order. To get into "Legacy" state,
you need to rollback Attachments first, then Project.
Also note that if Geo is enabled, after the migration was triggered, an event is generated
to replicate the operation on any Secondary node. That means the on disk changes will also
need to be performed on these nodes as well. Database changes will propagate without issues.
You must make sure the migration event was already processed or otherwise it may migrate
the files back to Hashed state again.
##### Attachments
To rollback single Attachment migration, rename `aa/bb/abcdef1234567890...` folder back to `namespace/project`.
Both folder names can be generated by the `FileUploader.absolute_base_dir(project)`, you
just need to switch the version from the `project` back to the previous one.
```ruby
project.storage_version
# => 2
FileUploader.absolute_base_dir(project)
# => "/opt/gitlab/embedded/service/gitlab-rails/public/uploads/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35"
project.storage_version = 1
FileUploader.absolute_base_dir(project)
# => "/opt/gitlab/embedded/service/gitlab-rails/public/uploads/gitlab/gitlab-shell-renamed"
```
##### Project
To rollback single Project migration, move `@hashed/aa/bb/aabbcdef1234567890abcdef.git` and `@hashed/aa/bb/aabbcdef1234567890abcdef.wiki.git`
back to `namespace/project.git` and `namespace/project.wiki.git` respectively and switch the version from the `project` back to `null`.
### Hashed Storage coverage
We are incrementally moving every storable object in GitLab to the Hashed
@ -100,6 +140,30 @@ which is true for CI Cache and LFS Objects.
| Pages | Yes | No | - | - |
| Docker Registry | Yes | No | - | - |
| CI Build Logs | No | No | - | - |
| CI Artifacts | No | No | Yes (Premium) | - |
| CI Artifacts | No | No | Yes | 9.4 / 10.6 |
| CI Cache | No | No | Yes | - |
| LFS Objects | Yes | No | Yes (Premium) | - |
| LFS Objects | Yes | Similar | Yes | 10.0 / 10.7 |
#### Implementation Details
##### Avatars
Each file is stored in a folder with its `id` from the database. The filename is always `avatar.png` for user avatars.
When avatar is replaced, `Upload` model is destroyed and a new one takes place with different `id`.
##### CI Artifacts
CI Artifacts are S3 compatible since **9.4** (GitLab Premium), and available in GitLab Core since **10.6**.
##### LFS Objects
LFS Objects implements a similar storage pattern using 2 chars, 2 level folders, following git own implementation:
```ruby
"shared/lfs-objects/#{oid[0..1}/#{oid[2..3]}/#{oid[4..-1]}"
# Based on object `oid`: `8909029eb962194cfb326259411b22ae3f4a814b5be4f80651735aeef9f3229c`, path will be:
"shared/lfs-objects/89/09/029eb962194cfb326259411b22ae3f4a814b5be4f80651735aeef9f3229c"
```
They are also S3 compatible since **10.0** (GitLab Premium), and available in GitLab Core since **10.7**.