kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Add latest changes from gitlab-org/gitlab@master

Browse source
This commit is contained in:
GitLab Bot 2022-05-18 06:08:06 +00:00
parent 8a735cc806
commit 7134f4d494
7 changed files with 153 additions and 104 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
doc/administration/geo/disaster_recovery/planned_failover.md
View file

@ -122,18 +122,24 @@ gitlab-rake gitlab:geo:check
If any failures are reported on either site, they should be resolved **before**
scheduling a planned failover.
### Check that secrets match between sites
### Check that secrets and SSH host keys match between nodes
The SSH host keys and `/etc/gitlab/gitlab-secrets.json` files should be
identical on all sites. Check this by running the following on all sites and
identical on all nodes. Check this by running the following on all nodes and
comparing the output:
```shell
sudo sha256sum /etc/ssh/ssh_host* /etc/gitlab/gitlab-secrets.json
```
If any files differ, replace the content on the **secondary** site with the
content from the **primary** site.
If any files differ, [manually replicate GitLab secrets](../replication/configuration.md#step-1-manually-replicate-secret-gitlab-values) and [replicate SSH host keys](../replication/configuration.md#step-2-manually-replicate-the-primary-sites-ssh-host-keys)
to the **secondary** site as necessary.
### Check that the correct certificates are installed for HTTPS
This step can be safely skipped if the **primary** site and all external sites accessed by the **primary** site use public CA-issued certificates.
If the **primary** site uses custom or self-signed TLS certificates to secure inbound connections or if the **primary** site connects to external services that use custom or self-signed certificates, the correct certificates should also be installed on the **secondary** site. Follow instructions for [using custom certificates](../replication/configuration.md#step-4-optional-using-custom-certificates) with **secondary** sites.
### Ensure Geo replication is up-to-date

2
doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md
View file

@ -84,7 +84,7 @@ failed to replicate is **lost**.
You can use the
[Geo status API](../../../../api/geo_nodes.md#retrieve-project-sync-or-verification-failures-that-occurred-on-the-current-node)
to review failed objects and the reasons for failure.
A common cause of replication failures is the data being missing on the
A common cause of replication failures is data that is missing on the
**primary** site - you can resolve these failures by restoring the data from backup,
or removing references to the missing data.

59
doc/administration/geo/replication/configuration.md
View file

@ -241,15 +241,60 @@ that the **secondary** site can act on those notifications immediately.
Be sure the _secondary_ site is running and accessible. You can sign in to the
_secondary_ site with the same credentials as were used with the _primary_ site.
### Step 4. (Optional) Configuring the **secondary** site to trust the **primary** site
### Step 4. (Optional) Using custom certificates
You can safely skip this step if your **primary** site uses a CA-issued HTTPS certificate.
You can safely skip this step if:
If your **primary** site is using a self-signed certificate for *HTTPS* support, you
need to add that certificate to the **secondary** site's trust store. Retrieve the
certificate from the **primary** site and follow
[these instructions](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates)
on the **secondary** site.
- Your **primary** site uses a public CA-issued HTTPS certificate.
- Your **primary** site only connects to external services with CA-issued (not self-signed) HTTPS certificates.
#### Custom or self-signed certificate for inbound connections
If your GitLab Geo **primary** site uses a custom or [self-signed certificate to secure inbound HTTPS connections](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates), this certificate can either be single-domain certificate or multi-domain.
Install the correct certificate based on your certificate type:
- **Multi-domain certificate** that includes both primary and secondary site domains: Install the certificate at `/etc/gitlab/ssl` on all **Rails, Sidekiq, and Gitaly** nodes in the **secondary** site.
- **Single-domain certificate** where the certificates are specific to each Geo site domain: Generate a valid certificate for your **secondary** site's domain and install it at `/etc/gitlab/ssl` per [these instructions](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates) on all **Rails, Sidekiq, and Gitaly** nodes in the **secondary** site.
#### Connecting to external services that use customer certificates
A copy of the self-signed certificate for the external service needs to be added to the trust store on all the **primary** site's nodes that require access to the service.
For the **secondary** site to be able to access the same external services, these certificates *must* be added to the **secondary** site's trust store.
If your **primary** site is using a [custom or self-signed certificate for inbound HTTPS connections](#custom-or-self-signed-certificate-for-inbound-connections), the **primary** site's certificate needs to be added to the **secondary** site's trust store:
1. SSH into each **Rails, Sidekiq, and Gitaly node on your secondary** site and login as root:
```shell
sudo -i
```
1. Copy the trusted certs from the **primary** site:
If you can access one of the nodes on your **primary** site serving SSH traffic using the root user:
```shell
scp root@<primary_site_node_fqdn>:/etc/gitlab/trusted-certs/* /etc/gitlab/trusted-certs
```
If you only have access through a user with sudo privileges:
```shell
# Run this from the node on your primary site:
sudo tar --transform 's/.*\///g' -zcvf ~/geo-trusted-certs.tar.gz /etc/gitlab/trusted-certs/*
# Run this on each node on your secondary site:
scp <user_with_sudo>@<primary_site_node_fqdn>:geo-trusted-certs.tar.gz .
tar zxvf ~/geo-trusted-certs.tar.gz -C /etc/gitlab/trusted-certs
```
1. Reconfigure each updated **Rails, Sidekiq, and Gitaly node in your secondary** site:
```shell
sudo gitlab-ctl reconfigure
```
### Step 5. Enable Git access over HTTP/HTTPS

93
doc/development/service_ping/index.md
View file

@ -163,7 +163,7 @@ We also collect metrics specific to [Geo](../../administration/geo/index.md) sec
]
```
### Enable or disable service ping metadata reporting
### Enable or disable service ping metadata reporting
Service Ping timing metadata reporting is under development but ready for production use.
It is deployed behind a feature flag that is **enabled by default**.
@ -444,7 +444,7 @@ To generate Service Ping, use [Teleport](https://goteleport.com/docs/) or a deta
```
1. Connect to console host:
```shell
ssh $USER-rails@console-01-sv-gprd.c.gitlab-production.internal
```
@ -457,14 +457,14 @@ To generate Service Ping, use [Teleport](https://goteleport.com/docs/) or a deta
1. To detach from screen, press `ctrl + A`, `ctrl + D`.
1. Exit from bastion:
```shell
exit
```
1. Get the metrics duration from logs:
Search in Google Console logs for `time_elapsed`. Query example [here](https://cloudlogging.app.goo.gl/nWheZvD8D3nWazNe6).
Search in Google Console logs for `time_elapsed`. Query example [here](https://cloudlogging.app.goo.gl/nWheZvD8D3nWazNe6).
### Verification (After approx 30 hours)
@ -495,7 +495,7 @@ Search in Google Console logs for `time_elapsed`. Query example [here](https://c
```
1. Check the last payload in `raw_usage_data` table:
```shell
RawUsageData.last.payload
```
@ -519,89 +519,6 @@ ServicePing::SubmitService.new(skip_db_write: true).execute
Service Ping reporting process state is monitored with [internal SiSense dashboard](https://app.periscopedata.com/app/gitlab/968489/Product-Intelligence---Service-Ping-Health).
## Troubleshooting
### Cannot disable Service Ping using the configuration file
The method to disable Service Ping using the GitLab configuration file does not work in
GitLab versions 9.3.0 to 13.12.3. To disable it, you must use the Admin Area in
the GitLab UI instead. For more information, see
[this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/333269).
GitLab functionality and application settings cannot override or circumvent
restrictions at the network layer. If Service Ping is blocked by your firewall,
you are not impacted by this bug.
#### Check if you are affected
You can check if you were affected by this bug by using the Admin Area or by
checking the configuration file of your GitLab instance:
- Using the Admin Area:
1. On the top bar, select **Menu > Admin**.
1. On the left sidebar, select **Settings > Metrics and profiling**.
1. Expand **Usage Statistics**.
1. Are you able to check or uncheck the checkbox to disable Service Ping?
- If _yes_, your GitLab instance is not affected by this bug.
- If you can't check or uncheck the checkbox, you are affected by this bug.
See the steps on [how to fix this](#how-to-fix-the-cannot-disable-service-ping-bug).
- Checking your GitLab instance configuration file:
To check whether you're impacted by this bug, check your instance configuration
settings. The configuration file in which Service Ping can be disabled depends
on your installation and deployment method, but is typically one of the following:
- `/etc/gitlab/gitlab.rb` for Omnibus GitLab Linux Package and Docker.
- `charts.yaml` for GitLab Helm and cloud-native Kubernetes deployments.
- `gitlab.yml` for GitLab installations from source.
To check the relevant configuration file for strings that indicate whether
Service Ping is disabled, you can use `grep`:
```shell
# Linux package
grep "usage_ping_enabled'\] = false" /etc/gitlab/gitlab.rb
# Kubernetes charts
grep "enableUsagePing: false" values.yaml
# From source
grep "usage_ping_enabled'\] = false" gitlab/config.yml
```
If you see any output after running the relevant command, your GitLab instance
may be affected by the bug. Otherwise, your instance is not affected.
#### How to fix the "Cannot disable Service Ping" bug
To work around this bug, you have two options:
- [Update](../../update/index.md) to GitLab 13.12.4 or newer to fix this bug.
- If you can't update to GitLab 13.12.4 or newer, enable Service Ping in the
configuration file, then disable Service Ping in the UI. For example, if you're
using the Linux package:
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['usage_ping_enabled'] = true
```
1. Reconfigure GitLab:
```shell
sudo gitlab-ctl reconfigure
```
1. In GitLab, on the top bar, select **Menu > Admin**.
1. On the left sidebar, select **Settings > Metrics and profiling**.
1. Expand **Usage Statistics**.
1. Clear the **Enable Service Ping** checkbox.
1. Select **Save Changes**.
## Related topics
- [Product Intelligence Guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/)

85
doc/development/service_ping/troubleshooting.md
View file

@ -22,10 +22,91 @@ The alert compares the current daily value with the daily value from previous we
You can use [this query](https://gitlab.com/gitlab-org/gitlab/-/issues/347298#note_836685350) as an example, to start detecting when the drop started.
### Troubleshooting GitLab application layer
### Troubleshoot the GitLab application layer
For results about an investigation conducted into an unexpected drop in Service ping Payload events volume, see [this issue](https://gitlab.com/gitlab-data/analytics/-/issues/11071).
### Troubleshooting data warehouse layer
### Troubleshoot the data warehouse layer
Reach out to the [Data team](https://about.gitlab.com/handbook/business-technology/data-team/) to ask about current state of data warehouse. On their handbook page there is a [section with contact details](https://about.gitlab.com/handbook/business-technology/data-team/#how-to-connect-with-us).
### Cannot disable Service Ping with the configuration file
The method to disable Service Ping with the GitLab configuration file does not work in
GitLab versions 9.3.0 to 13.12.3. To disable it, you must use the Admin Area in
the GitLab UI instead. For more information, see
[this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/333269).
GitLab functionality and application settings cannot override or circumvent
restrictions at the network layer. If Service Ping is blocked by your firewall,
you are not impacted by this bug.
#### Check if you are affected
You can check if you were affected by this bug by using the Admin Area or by
checking the configuration file of your GitLab instance:
- Using the Admin Area:
1. On the top bar, select **Menu > Admin**.
1. On the left sidebar, select **Settings > Metrics and profiling**.
1. Expand **Usage Statistics**.
1. Are you able to check or uncheck the checkbox to disable Service Ping?
- If _yes_, your GitLab instance is not affected by this bug.
- If you can't check or uncheck the checkbox, you are affected by this bug.
See the steps on [how to fix this](#how-to-fix-the-cannot-disable-service-ping-bug).
- Checking your GitLab instance configuration file:
To check whether you're impacted by this bug, check your instance configuration
settings. The configuration file in which Service Ping can be disabled depends
on your installation and deployment method, but is typically one of the following:
- `/etc/gitlab/gitlab.rb` for Omnibus GitLab Linux Package and Docker.
- `charts.yaml` for GitLab Helm and cloud-native Kubernetes deployments.
- `gitlab.yml` for GitLab installations from source.
To check the relevant configuration file for strings that indicate whether
Service Ping is disabled, you can use `grep`:
```shell
# Linux package
grep "usage_ping_enabled'\] = false" /etc/gitlab/gitlab.rb
# Kubernetes charts
grep "enableUsagePing: false" values.yaml
# From source
grep "usage_ping_enabled'\] = false" gitlab/config.yml
```
If you see any output after running the relevant command, your GitLab instance
may be affected by the bug. Otherwise, your instance is not affected.
#### How to fix the "Cannot disable Service Ping" bug
To work around this bug, you have two options:
- [Update](../../update/index.md) to GitLab 13.12.4 or newer to fix this bug.
- If you can't update to GitLab 13.12.4 or newer, enable Service Ping in the
configuration file, then disable Service Ping in the UI. For example, if you're
using the Linux package:
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['usage_ping_enabled'] = true
```
1. Reconfigure GitLab:
```shell
sudo gitlab-ctl reconfigure
```
1. In GitLab, on the top bar, select **Menu > Admin**.
1. On the left sidebar, select **Settings > Metrics and profiling**.
1. Expand **Usage Statistics**.
1. Clear the **Enable Service Ping** checkbox.
1. Select **Save Changes**.

2
doc/user/admin_area/settings/usage_statistics.md
View file

@ -123,7 +123,7 @@ To enable or disable Service Ping and version check:
NOTE:
The method to disable Service Ping in the GitLab configuration file does not work in
GitLab versions 9.3 to 13.12.3. For more information about how to disable it, see [troubleshooting](../../../development/service_ping/index.md#cannot-disable-service-ping-using-the-configuration-file).
GitLab versions 9.3 to 13.12.3. For more information about how to disable it, see [troubleshooting](../../../development/service_ping/troubleshooting.md#cannot-disable-service-ping-with-the-configuration-file).
To disable Service Ping and prevent it from being configured in the future through
the Admin Area:

2
spec/features/merge_request/user_edits_merge_request_spec.rb
View file

@ -92,7 +92,7 @@ RSpec.describe 'User edits a merge request', :js do
select2('merge-test', from: '#merge_request_target_branch')
click_button('Save changes')
expect(page).to have_content("Request to merge #{merge_request.source_branch} into merge-test")
expect(page).to have_content("requested to merge #{merge_request.source_branch} into merge-test")
expect(page).to have_content("changed target branch from #{merge_request.target_branch} to merge-test")
end