diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 32c2e4a5c99..56db7b5eb3a 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -409,7 +409,7 @@ an access key from the Google console first: 1. Select "Interoperability" and create an access key 1. Make note of the "Access Key" and "Secret" and replace them in the configurations below -1. In the buckets advanced settings ensure the Access Control option "Set object-level +1. In the buckets advanced settings ensure the Access Control option "Set object-level and bucket-level permissions" is selected 1. Make sure you already have a bucket created @@ -848,15 +848,24 @@ including (but not restricted to): * [Project mirroring](../workflow/repository_mirroring.md) * [Web hooks](../user/project/integrations/webhooks.md) -In the case of CI/CD, variables, you might experience some weird behavior, like -stuck jobs or 500 errors. In that case, you can try removing contents of the -`ci_group_variables` and `ci_project_variables` tables from the database. +In cases like CI/CD variables and Runner authentication, you might +experience some unexpected behavior such as: + +- Stuck jobs. +- 500 errors. + +In this case, you are required to reset all the tokens for CI/CD variables +and Runner Authentication, which is described in more detail below. After +resetting the tokens, you should be able to visit your project and the jobs +will have started running again. CAUTION: **Warning:** Use the following commands at your own risk, and make sure you've taken a backup beforehand. -1. Enter the Rails console: +#### Reset CI/CD variables + +1. Enter the DB console: For Omnibus GitLab packages: @@ -889,8 +898,39 @@ backup beforehand. 1. You may need to reconfigure or restart GitLab for the changes to take effect. -You should now be able to visit your project, and the jobs will start -running again. + +#### Reset Runner registration tokens + +1. Enter the DB console: + + For Omnibus GitLab packages: + + ```sh + sudo gitlab-rails dbconsole + ``` + + For installations from source: + + ```sh + sudo -u git -H bundle exec rails dbconsole RAILS_ENV=production + ``` + +1. Clear all the tokens for projects, groups, and the whole instance: + +CAUTION: **Caution:** +The last UPDATE operation will stop the runners being able to pick up +new jobs. You must register new runners. + + ```sql + -- Clear project tokens + UPDATE projects SET runners_token = null, runners_token_encrypted = null; + -- Clear group tokens + UPDATE namespaces SET runners_token = null, runners_token_encrypted = null; + -- Clear instance tokens + UPDATE application_settings SET runners_registration_token_encrypted = null; + -- Clear runner tokens + UPDATE ci_runners SET token = null, token_encrypted = null; + ``` A similar strategy can be employed for the remaining features - by removing the data that cannot be decrypted, GitLab can be brought back into working order,