kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Add latest changes from gitlab-org/gitlab@master

Browse source
This commit is contained in:
GitLab Bot 2022-11-01 09:10:48 +00:00
parent 38a4098900
commit 58c42f2f45
30 changed files with 340 additions and 373 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
config/metrics/counts_28d/20210216180745_action_monthly_active_users_design_management.yml
View file

@ -10,6 +10,10 @@ value_type: number
status: active
time_frame: 28d
data_source: redis_hll
instrumentation_class: RedisHLLMetric
options:
events:
- design_action
distribution:
- ce
- ee

4
config/metrics/counts_28d/20210216180747_action_monthly_active_users_wiki_repo.yml
View file

@ -10,6 +10,10 @@ value_type: number
status: active
time_frame: 28d
data_source: redis_hll
instrumentation_class: RedisHLLMetric
options:
events:
- wiki_action
distribution:
- ce
- ee

4
config/metrics/counts_28d/20210216182040_action_monthly_active_users_project_repo.yml
View file

@ -10,6 +10,10 @@ value_type: number
status: active
time_frame: 28d
data_source: redis_hll
instrumentation_class: RedisHLLMetric
options:
events:
- project_action
distribution:
- ce
- ee

4
config/metrics/counts_28d/20210216182041_action_monthly_active_users_git_write.yml
View file

@ -10,6 +10,10 @@ value_type: number
status: active
time_frame: 28d
data_source: redis_hll
instrumentation_class: RedisHLLMetric
options:
events:
- git_write_action
distribution:
- ce
- ee

6
doc/administration/auth/ldap/index.md
View file

@ -179,7 +179,7 @@ These configuration settings are available:
| `allow_username_or_email_login` | If enabled, GitLab ignores everything after the first `@` in the LDAP username submitted by the user on sign-in. If you are using `uid: 'userPrincipalName'` on ActiveDirectory you must disable this setting, because the userPrincipalName contains an `@`. | **{dotted-circle}** No | boolean |
| `block_auto_created_users` | To maintain tight control over the number of billable users on your GitLab installation, enable this setting to keep new users blocked until they have been cleared by an administrator (default: false). | **{dotted-circle}** No | boolean |
| `base` | Base where we can search for users. | **{check-circle}** Yes | `'ou=people,dc=gitlab,dc=example'` or `'DC=mydomain,DC=com'` |
| `user_filter` | Filter LDAP users. Format: [RFC 4515](https://tools.ietf.org/search/rfc4515) Note: GitLab does not support `omniauth-ldap`'s custom filter syntax. | **{dotted-circle}** No | For examples, read [Examples of user filters](#examples-of-user-filters). |
| `user_filter` | Filter LDAP users. Format: [RFC 4515](https://www.rfc-editor.org/rfc/rfc4515.html) Note: GitLab does not support `omniauth-ldap`'s custom filter syntax. | **{dotted-circle}** No | For examples, read [Examples of user filters](#examples-of-user-filters). |
| `lowercase_usernames` | If enabled, GitLab converts the name to lower case. | **{dotted-circle}** No | boolean |
| `retry_empty_result_with_codes` | An array of LDAP query response code that attempt to retry the operation if the result/content is empty. For Google Secure LDAP, set this value to `[80]`. | **{dotted-circle}** No | `[80]` |
@ -281,7 +281,7 @@ This example results in a sign-in page with the following tabs:
To limit all GitLab access to a subset of the LDAP users on your LDAP server, first narrow the
configured `base`. However, to further filter users if
necessary, you can set up an LDAP user filter. The filter must comply with [RFC 4515](https://tools.ietf.org/search/rfc4515).
necessary, you can set up an LDAP user filter. The filter must comply with [RFC 4515](https://www.rfc-editor.org/rfc/rfc4515.html).
- Example user filter for Omnibus GitLab instances:
@ -336,7 +336,7 @@ The `user_filter` DN can contain special characters. For example:
```
These characters must be escaped as documented in
[RFC 4515](https://tools.ietf.org/search/rfc4515).
[RFC 4515](https://www.rfc-editor.org/rfc/rfc4515.html).
- Escape commas with `\2C`. For example:

8
doc/administration/consul.md
View file

@ -8,7 +8,7 @@ type: reference
# How to set up Consul **(PREMIUM SELF)**
A Consul cluster consists of both
[server and client agents](https://www.consul.io/docs/agent).
[server and client agents](https://developer.hashicorp.com/consul/docs/agent).
The servers run on their own nodes and the clients run on other nodes that in
turn communicate with the servers.
@ -99,7 +99,7 @@ Consul nodes communicate using the raft protocol. If the current leader goes
offline, there must be a leader election. A leader node must exist to facilitate
synchronization across the cluster. If too many nodes go offline at the same time,
the cluster loses quorum and doesn't elect a leader due to
[broken consensus](https://www.consul.io/docs/architecture/consensus).
[broken consensus](https://developer.hashicorp.com/consul/docs/architecture/consensus).
Consult the [troubleshooting section](#troubleshooting-consul) if the cluster is not
able to recover after the upgrade. The [outage recovery](#outage-recovery) may
@ -148,7 +148,7 @@ you follow the Consul [outage recovery](#outage-recovery) process.
To be safe, it's recommended that you only restart Consul in one node at a time to
ensure the cluster remains intact. For larger clusters, it is possible to restart
multiple nodes at a time. See the
[Consul consensus document](https://www.consul.io/docs/architecture/consensus#deployment-table)
[Consul consensus document](https://developer.hashicorp.com/consul/docs/architecture/consensus#deployment-table)
for the number of failures it can tolerate. This is the number of simultaneous
restarts it can sustain.
@ -161,7 +161,7 @@ sudo gitlab-ctl restart consul
### Consul nodes unable to communicate
By default, Consul attempts to
[bind](https://www.consul.io/docs/agent/config/config-files#bind_addr) to `0.0.0.0`, but
[bind](https://developer.hashicorp.com/consul/docs/agent/config/config-files#bind_addr) to `0.0.0.0`, but
it advertises the first private IP address on the node for other Consul nodes
to communicate with it. If the other nodes cannot communicate with a node on
this address, then the cluster has a failed status.

4
doc/administration/monitoring/performance/grafana_configuration.md
View file

@ -133,8 +133,8 @@ However, you should **not** reinstate your old data _except_ under one of the fo
If you require access to your old Grafana data but don't meet one of these criteria, you may consider:
1. Reinstating it temporarily.
1. [Exporting the dashboards](https://grafana.com/docs/grafana/latest/dashboards/export-import/#exporting-a-dashboard) you need.
1. Refreshing the data and [re-importing your dashboards](https://grafana.com/docs/grafana/latest/dashboards/export-import/#import-dashboard).
1. [Exporting the dashboards](https://grafana.com/docs/grafana/latest/dashboards/manage-dashboards/#export-and-import-dashboards) you need.
1. Refreshing the data and [re-importing your dashboards](https://grafana.com/docs/grafana/latest/dashboards/manage-dashboards/#export-and-import-dashboards).
WARNING:
These actions pose a temporary vulnerability while your old Grafana data is in use.

2
doc/administration/package_information/defaults.md
View file

@ -70,6 +70,6 @@ over a network which will require, based on implementation, ports `111` and
NOTE:
In some cases, the GitLab Registry will be automatically enabled by default. See [our documentation](../packages/container_registry.md) for more details.
[^Consul-notes]: If using additional Consul functionality, more ports may need to be opened. See the [official documentation](https://www.consul.io/docs/install/ports#ports-table) for the list.
[^Consul-notes]: If using additional Consul functionality, more ports may need to be opened. See the [official documentation](https://developer.hashicorp.com/consul/docs/install/ports#ports-table) for the list.
[^Sidekiq-health]: If Sidekiq health check settings are not set, they will default to the Sidekiq metrics exporter settings. This default is deprecated and is set to be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/347509).

2
doc/api/index.md
View file

@ -593,7 +593,7 @@ GET /api/v4/projects/1/repository/tags/my%2Ftag
## Request Payload
API Requests can use parameters sent as [query strings](https://en.wikipedia.org/wiki/Query_string)
or as a [payload body](https://tools.ietf.org/html/draft-ietf-httpbis-p3-payload-14#section-3.2).
or as a [payload body](https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-p3-payload-14#section-3.2).
GET requests usually send a query string, while PUT or POST requests usually
send the payload body:

2
doc/api/oauth2.md
View file

@ -63,7 +63,7 @@ For a list of scopes in GitLab, see [the provider documentation](../integration/
### Prevent CSRF attacks
To [protect redirect-based flows](https://tools.ietf.org/id/draft-ietf-oauth-security-topics-13.html#rec_redirect),
To [protect redirect-based flows](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-security-topics-13#section-3.1),
the OAuth specification recommends the use of "One-time use CSRF tokens carried in the state
parameter, which are securely bound to the user agent", with each request to the
`/oauth/authorize` endpoint. This can prevent

2
doc/ci/testing/load_performance_testing.md
View file

@ -138,7 +138,7 @@ For example, you can override the duration of the test with a CLI option:
```
GitLab only displays the key performance metrics in the MR widget if k6's results are saved
via [summary export](https://k6.io/docs/results-visualization/json#summary-export)
via [summary export](https://k6.io/docs/results-output/real-time/json/#summary-export)
as a [Load Performance report artifact](../yaml/artifacts_reports.md#artifactsreportsload_performance).
The latest Load Performance artifact available is always used, using the
summary values from the test.

9
doc/development/database/query_recorder.md
View file

@ -39,12 +39,17 @@ end
As an example you might create 5 issues in between counts, which would cause the query count to increase by 5 if an N+1 problem exists.
In some cases the query count might change slightly between runs for unrelated reasons. In this case you might need to test `exceed_query_limit(control_count + acceptable_change)`, but this should be avoided if possible.
In some cases, the query count might change slightly between runs for unrelated reasons. In this case you might need to test `exceed_query_limit(control_count + acceptable_change)`, but this should be avoided if possible.
If this test fails, and the control was passed as a `QueryRecorder`, then the
failure message indicates where the extra queries are by matching queries on
the longest common prefix, grouping similar queries together.
In some cases, N+1 specs have been written to include three requests: first one to
warm the cache, second one to establish a control, third one to validate that
ther are no N+1 queries. Rather than make an extra request to warm the cache, prefer two requests
(control and test) and configure your test to ignore [cached queries](#cached-queries) in N+1 specs.
## Cached queries
By default, QueryRecorder ignores [cached queries](../merge_request_performance_guidelines.md#cached-queries) in the count. However, it may be better to count
@ -62,7 +67,7 @@ end
## Use request specs instead of controller specs
Use a [request spec](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/spec/requests) when writing a N+1 test on the controller level.
Use a [request spec](https://gitlab.com/gitlab-org/gitlab/-/tree/master/spec/requests) when writing a N+1 test on the controller level.
Controller specs should not be used to write N+1 tests as the controller is only initialized once per example.
This could lead to false successes where subsequent "requests" could have queries reduced (for example, because of memoization).

2
doc/development/fe_guide/style/scss.md
View file

@ -176,7 +176,7 @@ To check if any warnings are produced by your changes, run `yarn lint:stylelint`
catch any warnings.
If the Rake task is throwing warnings you don't understand, SCSS Lint's
documentation includes [a full list of their rules](https://stylelint.io/user-guide/rules/list/).
documentation includes [a full list of their rules](https://stylelint.io/user-guide/rules/).
### Fixing issues

6
doc/development/feature_flags/controls.md
View file

@ -16,7 +16,7 @@ To turn on/off features behind feature flags in any of the
GitLab-provided environments, like staging and production, you need to
have access to the [ChatOps](../chatops_on_gitlabcom.md) bot. The ChatOps bot
is currently running on the ops instance, which is different from
[GitLab.com](https://gitlab.com) or [`dev.gitlab.org`](https://dev.gitlab.org).
[GitLab.com](https://gitlab.com) or `dev.gitlab.org`.
Follow the ChatOps document to [request access](../chatops_on_gitlabcom.md#requesting-access).
@ -55,8 +55,8 @@ change feature flags or you do not have access.
### Enabling a feature for pre-production testing
As a first step in a feature rollout, you should enable the feature on
[`staging.gitlab.com`](https://staging.gitlab.com)
and [`dev.gitlab.org`](https://dev.gitlab.org).
`staging.gitlab.com`
and `dev.gitlab.org`.
These two environments have different scopes.
`dev.gitlab.org` is a production CE environment that has internal GitLab Inc.

2
doc/development/feature_flags/index.md
View file

@ -435,7 +435,7 @@ For example, the following feature flags are enabled for a certain percentage of
If a project A has `:feature-set-1` enabled, there is no guarantee that project A also has `:feature-set-2` enabled.
For more detail, see [This is how percentages work in Flipper](https://www.hackwithpassion.com/this-is-how-percentages-work-in-flipper).
For more detail, see [This is how percentages work in Flipper](https://www.hackwithpassion.com/this-is-how-percentages-work-in-flipper/).
#### Use actors for verifying in production

2
doc/development/secure_coding_guidelines.md
View file

@ -856,7 +856,7 @@ Working with archive files like `zip`, `tar`, `jar`, `war`, `cpio`, `apk`, `rar`
### Zip Slip
In 2018, the security company Snyk [released a blog post](https://snyk.io/research/zip-slip-vulnerability) describing research into a widespread and critical vulnerability present in many libraries and applications which allows an attacker to overwrite arbitrary files on the server file system which, in many cases, can be leveraged to achieve remote code execution. The vulnerability was dubbed Zip Slip.
In 2018, the security company Snyk [released a blog post](https://security.snyk.io/research/zip-slip-vulnerability) describing research into a widespread and critical vulnerability present in many libraries and applications which allows an attacker to overwrite arbitrary files on the server file system which, in many cases, can be leveraged to achieve remote code execution. The vulnerability was dubbed Zip Slip.
A Zip Slip vulnerability happens when an application extracts an archive without validating and sanitizing the filenames inside the archive for directory traversal sequences that change the file location when the file is extracted.

2
doc/development/service_ping/implement.md
View file

@ -266,7 +266,7 @@ Arguments:
#### Ordinary Redis counters
Example of implementation: [`Gitlab::UsageDataCounters::WikiPageCounter`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/wiki_page_counter.rb), using Redis methods [`INCR`](https://redis.io/commands/incr) and [`GET`](https://redis.io/commands/get).
Example of implementation: [`Gitlab::UsageDataCounters::WikiPageCounter`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/wiki_page_counter.rb), using Redis methods [`INCR`](https://redis.io/commands/incr/) and [`GET`](https://redis.io/commands/get/).
Events are handled by counter classes in the `Gitlab::UsageDataCounters` namespace, inheriting from `BaseCounter`, that are either:

2
doc/development/snowplow/index.md
View file

@ -186,7 +186,7 @@ LIMIT 100
### Web-specific parameters
Snowplow JavaScript adds [web-specific parameters](https://docs.snowplowanalytics.com/docs/collecting-data/collecting-from-own-applications/snowplow-tracker-protocol/#Web-specific_parameters) to all web events by default.
Snowplow JavaScript adds [web-specific parameters](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/snowplow-tracker-protocol/#Web-specific_parameters) to all web events by default.
## Related topics

4
doc/development/stage_group_observability/dashboards/index.md
View file

@ -43,7 +43,7 @@ All metrics recorded in the GitLab production system have
[one-year retention](https://gitlab.com/gitlab-cookbooks/gitlab-prometheus/-/blob/31526b03fef823e2f9b3cda7c75dcd28a12418a3/attributes/prometheus.rb#L40).
You can also zoom in and filter the time range directly on a graph. For more information, see the
[Grafana Time Range Controls](https://grafana.com/docs/grafana/latest/dashboards/time-range-controls/)
[Grafana Time Range Controls](https://grafana.com/docs/grafana/latest/dashboards/use-dashboards/#set-dashboard-time-range)
documentation.
## Filters and annotations
@ -51,7 +51,7 @@ documentation.
On each dashboard, there are two filters and some annotation switches on the top of the page.
Some special events are meaningful to development and operational activities.
[Grafana annotations](https://grafana.com/docs/grafana/latest/dashboards/annotations/) mark them
[Grafana annotations](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/annotate-visualizations/) mark them
directly on the graphs.
![Filters and annotations](img/stage_group_dashboards_filters.png)

2
doc/integration/gitpod.md
View file

@ -44,7 +44,7 @@ With the Gitpod integration enabled for your GitLab instance, to enable it for y
For GitLab self-managed instances, a GitLab administrator needs to:
1. Set up a Gitpod instance to integrate with GitLab. Refer to the [Gitpod documentation](https://www.gitpod.io/docs/self-hosted/latest)
1. Set up a Gitpod instance to integrate with GitLab. Refer to the [Gitpod documentation](https://www.gitpod.io/docs/configure/self-hosted/latest)
to get your instance up and running.
1. Enable it in GitLab:
1. On the top bar, select **Main menu > Admin**.

2
doc/user/infrastructure/iac/terraform_state.md
View file

@ -59,7 +59,7 @@ encrypt plan output or modify the project visibility settings.
To configure GitLab CI/CD as a backend:
1. In your Terraform project, in a `.tf` file like `backend.tf`,
define the [HTTP backend](https://www.terraform.io/docs/language/settings/backends/http.html):
define the [HTTP backend](https://developer.hashicorp.com/terraform/language/settings/backends/http):
```hcl
terraform {

2
doc/user/project/import/clearcase.md
View file

@ -27,7 +27,7 @@ The following table illustrates the main differences between ClearCase and Git:
| Server | UNIX, Windows legacy systems | UNIX, macOS |
| License | Proprietary | GPL |
_Taken from the slides [ClearCase and the journey to Git](https://docplayer.net/42708453-Clearcase-the-journey-to-git-migrating-your-skills-and-vobs-to-git.html) provided by [collab.net](https://www.collab.net/)_
_Taken from the slides [ClearCase and the journey to Git](https://docplayer.net/42708453-Clearcase-the-journey-to-git-migrating-your-skills-and-vobs-to-git.html) provided by `collab.net`_
## Why migrate

2
doc/user/project/pages/public_folder.md
View file

@ -116,7 +116,7 @@ GitLab Pages supports only static sites.
```
1. Configure your Nuxt.js application for
[Static Site Generation](https://nuxtjs.org/docs/features/deployment-targets#static-hosting).
[Static Site Generation](https://nuxtjs.org/docs/features/deployment-targets/#static-hosting).
### Vite

33
lib/gitlab/usage_data.rb
View file

@ -621,10 +621,15 @@ module Gitlab
end
def action_monthly_active_users(time_period)
counter = Gitlab::UsageDataCounters::EditorUniqueCounter
date_range = { date_from: time_period[:created_at].first, date_to: time_period[:created_at].last }
event_monthly_active_users(date_range)
.merge!(ide_monthly_active_users(date_range))
{
action_monthly_active_users_web_ide_edit: redis_usage_data { counter.count_web_ide_edit_actions(**date_range) },
action_monthly_active_users_sfe_edit: redis_usage_data { counter.count_sfe_edit_actions(**date_range) },
action_monthly_active_users_snippet_editor_edit: redis_usage_data { counter.count_snippet_editor_edit_actions(**date_range) },
action_monthly_active_users_ide_edit: redis_usage_data { counter.count_edit_using_editor(**date_range) }
}
end
def with_duration
@ -682,30 +687,6 @@ module Gitlab
end
end
def event_monthly_active_users(date_range)
data = {
action_monthly_active_users_project_repo: Gitlab::UsageDataCounters::TrackUniqueEvents::PUSH_ACTION,
action_monthly_active_users_design_management: Gitlab::UsageDataCounters::TrackUniqueEvents::DESIGN_ACTION,
action_monthly_active_users_wiki_repo: Gitlab::UsageDataCounters::TrackUniqueEvents::WIKI_ACTION,
action_monthly_active_users_git_write: Gitlab::UsageDataCounters::TrackUniqueEvents::GIT_WRITE_ACTION
}
data.each do |key, event|
data[key] = redis_usage_data { Gitlab::UsageDataCounters::TrackUniqueEvents.count_unique_events(event_action: event, **date_range) }
end
end
def ide_monthly_active_users(date_range)
counter = Gitlab::UsageDataCounters::EditorUniqueCounter
{
action_monthly_active_users_web_ide_edit: redis_usage_data { counter.count_web_ide_edit_actions(**date_range) },
action_monthly_active_users_sfe_edit: redis_usage_data { counter.count_sfe_edit_actions(**date_range) },
action_monthly_active_users_snippet_editor_edit: redis_usage_data { counter.count_snippet_editor_edit_actions(**date_range) },
action_monthly_active_users_ide_edit: redis_usage_data { counter.count_edit_using_editor(**date_range) }
}
end
def distinct_count_service_desk_enabled_projects(time_period)
project_creator_id_start = minimum_id(User)
project_creator_id_finish = maximum_id(User)

369
qa/qa/specs/features/api/1_manage/import/import_github_repo_spec.rb
View file

@ -1,194 +1,227 @@
# frozen_string_literal: true
module QA
# https://github.com/gitlab-qa-github/import-test <- project under test
# Spec uses real github.com, which means outage of github.com can actually block deployment
# Keep spec in reliable bucket but don't run in blocking pipelines
#
RSpec.describe 'Manage', product_group: :import do
describe 'GitHub import', :reliable do
include_context 'with github import'
context 'when imported via api' do
it 'imports project', testcase: 'https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347670' do
imported_project.reload! # import the project
expect { imported_project.project_import_status[:import_status] }.to eventually_eq('finished')
.within(max_duration: 240, sleep_interval: 1)
aggregate_failures do
verify_status_data
verify_repository_import
verify_protected_branches_import
verify_commits_import
verify_labels_import
verify_issues_import
verify_milestones_import
verify_wikis_import
verify_merge_requests_import
verify_release_import
end
# https://github.com/gitlab-qa-github/import-test <- project under test
RSpec.describe 'Manage', :github, :reliable, :skip_live_env, :requires_admin, product_group: :import do
describe 'Project import', issue: 'https://gitlab.com/gitlab-org/gitlab/-/issues/353583' do
let!(:api_client) { Runtime::API::Client.as_admin }
let!(:group) { Resource::Group.fabricate_via_api! { |resource| resource.api_client = api_client } }
let!(:user) do
Resource::User.fabricate_via_api! do |resource|
resource.api_client = api_client
resource.hard_delete_on_api_removal = true
end
end
def verify_status_data
stats = imported_project.project_import_status.dig(:stats, :imported)
expect(stats).to include(
issue: 1,
label: 9,
milestone: 1,
note: 3,
pull_request: 1,
pull_request_review: 1,
diff_note: 1,
release: 1
)
let!(:user_api_client) { Runtime::API::Client.new(user: user) }
let(:imported_project) do
Resource::ProjectImportedFromGithub.fabricate_via_api! do |project|
project.name = 'imported-project'
project.group = group
project.github_personal_access_token = Runtime::Env.github_access_token
project.github_repository_path = 'gitlab-qa-github/import-test'
project.api_client = user_api_client
project.issue_events_import = true
project.full_notes_import = true
end
end
def verify_repository_import
expect(imported_project.reload!.description).to eq('Project for github import test')
expect(imported_project.api_response[:import_error]).to be_nil
before do
group.add_member(user, Resource::Members::AccessLevel::MAINTAINER)
end
after do
user.remove_via_api!
end
it 'imports Github repo via api', testcase: 'https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347670' do
imported_project.reload! # import the project
expect { imported_project.project_import_status[:import_status] }.to eventually_eq('finished')
.within(max_duration: 240, sleep_interval: 1)
aggregate_failures do
verify_status_data
verify_repository_import
verify_protected_branches_import
verify_commits_import
verify_labels_import
verify_issues_import
verify_milestones_import
verify_wikis_import
verify_merge_requests_import
verify_release_import
end
end
def verify_protected_branches_import
branches = imported_project.protected_branches.map do |branch|
branch.slice(:name, :allow_force_push, :code_owner_approval_required)
end
expect(branches.first).to include(
{
name: 'main'
# TODO: Add validation once https://gitlab.com/groups/gitlab-org/-/epics/8585 is closed
# At the moment both options are always set to false regardless of state in github
# allow_force_push: true,
# code_owner_approval_required: true
}
)
def verify_status_data
stats = imported_project.project_import_status.dig(:stats, :imported)
expect(stats).to include(
issue: 1,
label: 9,
milestone: 1,
note: 3,
pull_request: 1,
pull_request_review: 1,
diff_note: 1,
release: 1
)
end
def verify_repository_import
expect(imported_project.reload!.description).to eq('Project for github import test')
expect(imported_project.api_response[:import_error]).to be_nil
end
def verify_protected_branches_import
branches = imported_project.protected_branches.map do |branch|
branch.slice(:name, :allow_force_push, :code_owner_approval_required)
end
expect(branches.first).to include(
{
name: 'main'
# TODO: Add validation once https://gitlab.com/groups/gitlab-org/-/epics/8585 is closed
# At the moment both options are always set to false regardless of state in github
# allow_force_push: true,
# code_owner_approval_required: true
}
)
def verify_commits_import
expect(imported_project.commits.length).to eq(2)
end
# GitHub branch protection rule "Require signed commits" is mapped to the
# "Reject unsigned commits" push rule
expect(imported_project.push_rules[:reject_unsigned_commits]).to be_truthy
end
def verify_labels_import
labels = imported_project.labels.map { |label| label.slice(:name, :color) }
def verify_commits_import
expect(imported_project.commits.length).to eq(2)
end
expect(labels).to include(
{ name: 'bug', color: '#d73a4a' },
{ name: 'documentation', color: '#0075ca' },
{ name: 'duplicate', color: '#cfd3d7' },
{ name: 'enhancement', color: '#a2eeef' },
{ name: 'good first issue', color: '#7057ff' },
{ name: 'help wanted', color: '#008672' },
{ name: 'invalid', color: '#e4e669' },
{ name: 'question', color: '#d876e3' },
{ name: 'wontfix', color: '#ffffff' }
)
end
def verify_labels_import
labels = imported_project.labels.map { |label| label.slice(:name, :color) }
def verify_milestones_import
milestones = imported_project.milestones
expect(labels).to include(
{ name: 'bug', color: '#d73a4a' },
{ name: 'documentation', color: '#0075ca' },
{ name: 'duplicate', color: '#cfd3d7' },
{ name: 'enhancement', color: '#a2eeef' },
{ name: 'good first issue', color: '#7057ff' },
{ name: 'help wanted', color: '#008672' },
{ name: 'invalid', color: '#e4e669' },
{ name: 'question', color: '#d876e3' },
{ name: 'wontfix', color: '#ffffff' }
)
end
expect(milestones.length).to eq(1)
expect(milestones.first).to include(title: '0.0.1', description: nil, state: 'active')
end
def verify_milestones_import
milestones = imported_project.milestones
def verify_wikis_import
wikis = imported_project.wikis
expect(milestones.length).to eq(1)
expect(milestones.first).to include(title: '0.0.1', description: nil, state: 'active')
end
expect(wikis.length).to eq(1)
expect(wikis.first).to include(title: 'Home', format: 'markdown')
end
def verify_wikis_import
wikis = imported_project.wikis
def verify_issues_import
issues = imported_project.issues
issue = Resource::Issue.init do |resource|
resource.project = imported_project
resource.iid = issues.first[:iid]
resource.api_client = user_api_client
end.reload!
comments, events = fetch_events_and_comments(issue)
expect(wikis.length).to eq(1)
expect(wikis.first).to include(title: 'Home', format: 'markdown')
end
expect(issues.length).to eq(1)
expect(issue.api_resource).to include(
title: 'Test issue',
description: "*Created by: gitlab-qa-github*\n\nTest issue description",
labels: ['good first issue', 'help wanted', 'question']
)
expect(comments).to match_array(
[
"*Created by: gitlab-qa-github*\n\nSome test comment",
"*Created by: gitlab-qa-github*\n\nAnother test comment"
]
)
expect(events).to match_array(
[
{ name: "add_label", label: "question" },
{ name: "add_label", label: "good first issue" },
{ name: "add_label", label: "help wanted" },
{ name: "add_milestone", label: "0.0.1" },
{ name: "closed" },
{ name: "reopened" }
]
)
end
def verify_issues_import
issues = imported_project.issues
issue = Resource::Issue.init do |resource|
resource.project = imported_project
resource.iid = issues.first[:iid]
resource.api_client = user_api_client
end.reload!
comments, events = fetch_events_and_comments(issue)
def verify_merge_requests_import
merge_requests = imported_project.merge_requests
merge_request = Resource::MergeRequest.init do |mr|
mr.project = imported_project
mr.iid = merge_requests.first[:iid]
mr.api_client = user_api_client
end.reload!
comments, events = fetch_events_and_comments(merge_request)
expect(merge_requests.length).to eq(1)
expect(merge_request.api_resource).to include(
title: 'Test pull request',
state: 'opened',
target_branch: 'main',
source_branch: 'gitlab-qa-github-patch-1',
labels: %w[documentation],
description: "*Created by: gitlab-qa-github*\n\nTest pull request body"
)
expect(comments).to match_array(
[
"*Created by: gitlab-qa-github*\n\n**Review:** Commented\n\nGood but needs some improvement",
"*Created by: gitlab-qa-github*\n\n```suggestion:-0+0\nProject for GitHub import test to GitLab\r\n```",
"*Created by: gitlab-qa-github*\n\nSome test PR comment"
]
)
expect(events).to match_array(
[
{ name: "add_label", label: "documentation" },
{ name: "add_milestone", label: "0.0.1" }
]
)
end
def verify_release_import
releases = imported_project.releases
expect(releases.length).to eq(1)
expect(releases.first).to include(
tag_name: "0.0.1",
name: "0.0.1",
description: "Initial release",
created_at: "2022-03-07T07:59:22.000Z",
released_at: "2022-03-07T08:02:09.000Z"
)
end
# Fetch events and comments from issue or mr
#
# @param [QA::Resource::Issuable] issuable
# @return [Array]
def fetch_events_and_comments(issuable)
comments = issuable.comments.map { |comment| comment[:body] }
events = [
*issuable.label_events.map { |e| { name: "#{e[:action]}_label", label: e.dig(:label, :name) } },
*issuable.state_events.map { |e| { name: e[:state] } },
*issuable.milestone_events.map { |e| { name: "#{e[:action]}_milestone", label: e.dig(:milestone, :title) } }
expect(issues.length).to eq(1)
expect(issue.api_resource).to include(
title: 'Test issue',
description: "*Created by: gitlab-qa-github*\n\nTest issue description",
labels: ['good first issue', 'help wanted', 'question']
)
expect(comments).to match_array(
[
"*Created by: gitlab-qa-github*\n\nSome test comment",
"*Created by: gitlab-qa-github*\n\nAnother test comment"
]
)
expect(events).to match_array(
[
{ name: "add_label", label: "question" },
{ name: "add_label", label: "good first issue" },
{ name: "add_label", label: "help wanted" },
{ name: "add_milestone", label: "0.0.1" },
{ name: "closed" },
{ name: "reopened" }
]
)
end
[comments, events]
end
def verify_merge_requests_import
merge_requests = imported_project.merge_requests
merge_request = Resource::MergeRequest.init do |mr|
mr.project = imported_project
mr.iid = merge_requests.first[:iid]
mr.api_client = user_api_client
end.reload!
comments, events = fetch_events_and_comments(merge_request)
expect(merge_requests.length).to eq(1)
expect(merge_request.api_resource).to include(
title: 'Test pull request',
state: 'opened',
target_branch: 'main',
source_branch: 'gitlab-qa-github-patch-1',
labels: %w[documentation],
description: "*Created by: gitlab-qa-github*\n\nTest pull request body"
)
expect(comments).to match_array(
[
"*Created by: gitlab-qa-github*\n\n**Review:** Commented\n\nGood but needs some improvement",
"*Created by: gitlab-qa-github*\n\n```suggestion:-0+0\nProject for GitHub import test to GitLab\r\n```",
"*Created by: gitlab-qa-github*\n\nSome test PR comment"
]
)
expect(events).to match_array(
[
{ name: "add_label", label: "documentation" },
{ name: "add_milestone", label: "0.0.1" }
]
)
end
def verify_release_import
releases = imported_project.releases
expect(releases.length).to eq(1)
expect(releases.first).to include(
tag_name: "0.0.1",
name: "0.0.1",
description: "Initial release",
created_at: "2022-03-07T07:59:22.000Z",
released_at: "2022-03-07T08:02:09.000Z"
)
end
# Fetch events and comments from issue or mr
#
# @param [QA::Resource::Issuable] issuable
# @return [Array]
def fetch_events_and_comments(issuable)
comments = issuable.comments.map { |comment| comment[:body] }
events = [
*issuable.label_events.map { |e| { name: "#{e[:action]}_label", label: e.dig(:label, :name) } },
*issuable.state_events.map { |e| { name: e[:state] } },
*issuable.milestone_events.map { |e| { name: "#{e[:action]}_milestone", label: e.dig(:milestone, :title) } }
]
[comments, events]
end
end
end

154
qa/qa/specs/features/browser_ui/1_manage/project/import_github_repo_spec.rb
View file

@ -4,90 +4,88 @@ module QA
# Spec uses real github.com, which means outage of github can actually block deployment
# Keep spec in reliable bucket but don't run in blocking pipelines
RSpec.describe 'Manage', :github, :reliable, :skip_live_env, :requires_admin, product_group: :import do
describe 'GitHub import' do
context 'when imported via UI' do
let(:github_repo) { 'gitlab-qa-github/import-test' }
let(:api_client) { Runtime::API::Client.as_admin }
let(:group) { Resource::Group.fabricate_via_api! { |resource| resource.api_client = api_client } }
let(:user) do
Resource::User.fabricate_via_api! do |resource|
resource.api_client = api_client
resource.hard_delete_on_api_removal = true
describe 'Project import' do
let(:github_repo) { 'gitlab-qa-github/import-test' }
let(:api_client) { Runtime::API::Client.as_admin }
let(:group) { Resource::Group.fabricate_via_api! { |resource| resource.api_client = api_client } }
let(:user) do
Resource::User.fabricate_via_api! do |resource|
resource.api_client = api_client
resource.hard_delete_on_api_removal = true
end
end
let(:imported_project) do
Resource::ProjectImportedFromGithub.init do |project|
project.import = true
project.group = group
project.github_personal_access_token = Runtime::Env.github_access_token
project.github_repository_path = github_repo
project.api_client = api_client
end
end
let(:imported_issue) do
Resource::Issue.init do |resource|
resource.project = imported_project
resource.iid = imported_project.issues.first[:iid]
resource.api_client = api_client
end.reload!
end
let(:imported_issue_events) do
imported_issue.label_events.map { |e| { name: "#{e[:action]}_label", label: e.dig(:label, :name) } }
end
before do
group.add_member(user, Resource::Members::AccessLevel::MAINTAINER)
Flow::Login.sign_in(as: user)
Page::Main::Menu.perform(&:go_to_create_project)
Page::Project::New.perform do |project_page|
project_page.click_import_project
project_page.click_github_link
end
end
after do
user.remove_via_api!
end
it 'imports a GitHub repo', testcase: 'https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347877' do
Page::Project::Import::Github.perform do |import_page|
import_page.add_personal_access_token(Runtime::Env.github_access_token)
import_page.select_advanced_option(:single_endpoint_issue_events_import)
import_page.select_advanced_option(:single_endpoint_notes_import)
import_page.select_advanced_option(:attachments_import)
import_page.import!(github_repo, group.full_path, imported_project.name)
aggregate_failures do
expect(import_page).to have_imported_project(github_repo, wait: 240)
# validate button is present instead of navigating to avoid dealing with multiple tabs
# which makes the test more complicated
expect(import_page).to have_go_to_project_button(github_repo)
end
end
let(:imported_project) do
Resource::ProjectImportedFromGithub.init do |project|
project.import = true
project.group = group
project.github_personal_access_token = Runtime::Env.github_access_token
project.github_repository_path = github_repo
project.api_client = api_client
imported_project.reload!.visit!
Page::Project::Show.perform do |project|
aggregate_failures do
expect(project).to have_content(imported_project.name)
expect(project).to have_content('Project for github import test')
end
end
let(:imported_issue) do
Resource::Issue.init do |resource|
resource.project = imported_project
resource.iid = imported_project.issues.first[:iid]
resource.api_client = api_client
end.reload!
end
let(:imported_issue_events) do
imported_issue.label_events.map { |e| { name: "#{e[:action]}_label", label: e.dig(:label, :name) } }
end
before do
group.add_member(user, Resource::Members::AccessLevel::MAINTAINER)
Flow::Login.sign_in(as: user)
Page::Main::Menu.perform(&:go_to_create_project)
Page::Project::New.perform do |project_page|
project_page.click_import_project
project_page.click_github_link
end
end
after do
user.remove_via_api!
end
it 'imports a project', testcase: 'https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347877' do
Page::Project::Import::Github.perform do |import_page|
import_page.add_personal_access_token(Runtime::Env.github_access_token)
import_page.select_advanced_option(:single_endpoint_issue_events_import)
import_page.select_advanced_option(:single_endpoint_notes_import)
import_page.select_advanced_option(:attachments_import)
import_page.import!(github_repo, group.full_path, imported_project.name)
aggregate_failures do
expect(import_page).to have_imported_project(github_repo, wait: 240)
# validate button is present instead of navigating to avoid dealing with multiple tabs
# which makes the test more complicated
expect(import_page).to have_go_to_project_button(github_repo)
end
end
imported_project.reload!.visit!
Page::Project::Show.perform do |project|
aggregate_failures do
expect(project).to have_content(imported_project.name)
expect(project).to have_content('Project for github import test')
end
end
# Validate :single_endpoint_issue_events_import option was triggered correctly and imported the events
expect(imported_issue_events).to match_array(
[
{ name: "add_label", label: "question" },
{ name: "add_label", label: "good first issue" },
{ name: "add_label", label: "help wanted" }
]
)
end
# Validate :single_endpoint_issue_events_import option was triggered correctly and imported the events
expect(imported_issue_events).to match_array(
[
{ name: "add_label", label: "question" },
{ name: "add_label", label: "good first issue" },
{ name: "add_label", label: "help wanted" }
]
)
end
end
end

43
qa/qa/specs/features/shared_contexts/github_import_shared_context.rb
View file

@ -1,43 +0,0 @@
# frozen_string_literal: true
module QA
RSpec.shared_context "with github import", :github, :skip_live_env, :requires_admin do
let!(:api_client) { Runtime::API::Client.as_admin }
let!(:group) do
Resource::Group.fabricate_via_api! do |resource|
resource.api_client = api_client
resource.path = "destination-group-for-import-#{SecureRandom.hex(4)}"
end
end
let!(:user) do
Resource::User.fabricate_via_api! do |resource|
resource.api_client = api_client
resource.hard_delete_on_api_removal = true
end
end
let!(:user_api_client) { Runtime::API::Client.new(user: user) }
let(:imported_project) do
Resource::ProjectImportedFromGithub.fabricate_via_api! do |project|
project.name = 'imported-project'
project.group = group
project.github_personal_access_token = Runtime::Env.github_access_token
project.github_repository_path = 'gitlab-qa-github/import-test'
project.api_client = user_api_client
project.issue_events_import = true
project.full_notes_import = true
end
end
before do
group.add_member(user, Resource::Members::AccessLevel::MAINTAINER)
end
after do
user.remove_via_api!
end
end
end

18
spec/lib/gitlab/usage_data_spec.rb
View file

@ -1130,20 +1130,6 @@ RSpec.describe Gitlab::UsageData, :aggregate_failures do
let(:project) { build(:project) }
before do
counter = Gitlab::UsageDataCounters::TrackUniqueEvents
project_type = Event::TARGET_TYPES[:project]
wiki = Event::TARGET_TYPES[:wiki]
design = Event::TARGET_TYPES[:design]
counter.track_event(event_action: :pushed, event_target: project_type, author_id: 1)
counter.track_event(event_action: :pushed, event_target: project_type, author_id: 1)
counter.track_event(event_action: :pushed, event_target: project_type, author_id: 2)
counter.track_event(event_action: :pushed, event_target: project_type, author_id: 3)
counter.track_event(event_action: :pushed, event_target: project_type, author_id: 4, time: time - 3.days)
counter.track_event(event_action: :created, event_target: wiki, author_id: 3)
counter.track_event(event_action: :created, event_target: design, author_id: 3)
counter.track_event(event_action: :created, event_target: design, author_id: 4)
counter = Gitlab::UsageDataCounters::EditorUniqueCounter
counter.track_web_ide_edit_action(author: user1, project: project)
@ -1162,10 +1148,6 @@ RSpec.describe Gitlab::UsageData, :aggregate_failures do
it 'returns the distinct count of user actions within the specified time period' do
expect(described_class.action_monthly_active_users(time_period)).to eq(
{
action_monthly_active_users_design_management: 2,
action_monthly_active_users_project_repo: 3,
action_monthly_active_users_wiki_repo: 1,
action_monthly_active_users_git_write: 4,
action_monthly_active_users_web_ide_edit: 2,
action_monthly_active_users_sfe_edit: 2,
action_monthly_active_users_snippet_editor_edit: 2,

9
spec/requests/api/groups_spec.rb
View file

@ -91,11 +91,8 @@ RSpec.describe API::Groups do
.to satisfy_one { |group| group['name'] == group1.name }
end
it 'avoids N+1 queries' do
# Establish baseline
get api("/groups", admin)
control = ActiveRecord::QueryRecorder.new do
it 'avoids N+1 queries', :use_sql_query_cache do
control = ActiveRecord::QueryRecorder.new(skip_cached: false) do
get api("/groups", admin)
end
@ -103,7 +100,7 @@ RSpec.describe API::Groups do
expect do
get api("/groups", admin)
end.not_to exceed_query_limit(control)
end.not_to exceed_all_query_limit(control)
end
context 'when statistics are requested' do

8
spec/requests/api/projects_spec.rb
View file

@ -169,10 +169,8 @@ RSpec.describe API::Projects do
shared_examples_for 'projects response without N + 1 queries' do |threshold|
let(:additional_project) { create(:project, :public) }
it 'avoids N + 1 queries' do
get api('/projects', current_user)
control = ActiveRecord::QueryRecorder.new do
it 'avoids N + 1 queries', :use_sql_query_cache do
control = ActiveRecord::QueryRecorder.new(skip_cached: false) do
get api('/projects', current_user)
end
@ -180,7 +178,7 @@ RSpec.describe API::Projects do
expect do
get api('/projects', current_user)
end.not_to exceed_query_limit(control).with_threshold(threshold)
end.not_to exceed_all_query_limit(control).with_threshold(threshold)
end
end