Add latest changes from gitlab-org/gitlab@master

.gitlab/issue_templates/Security developer workflow.md

@@ -36,7 +36,8 @@ After your merge request has been approved according to our [approval guidelines
 ## Documentation and final details
 
 - [ ] Ensure the [Links section](#links) is completed.
-- [ ] Find out the versions affected (the Git history of the files affected may help you with this) and add them to the [details section](#details)
+- [ ] Find out the [versions affected](https://gitlab.com/gitlab-org/release/docs/-/blob/master/general/security/developer.md#versions-affected) and add them to the [details section](#details)
+  * The Git history of the files affected may help you associate the issue with a [release](https://about.gitlab.com/releases/)
 - [ ] Fill in any upgrade notes that users may need to take into account in the [details section](#details)
 - [ ] Add Yes/No and further details if needed to the migration and settings columns in the [details section](#details)
 - [ ] Add the nickname of the external user who found the issue (and/or HackerOne profile) to the Thanks row in the [details section](#details)

CHANGELOG-EE.md

@@ -1,5 +1,48 @@
 Please view this file on the master branch, on stable branches it's out of date.
 
+## 12.10.0 (2020-04-22)
+
+### Fixed (6 changes, 1 of them is from the community)
+
+- Split CI minutes resets into different workers. !29017
+- Fix duplicated events on Value Stream Analytics stage form. !29030
+- Fix missing autocomplete results in the ElasticSearch admin UI. !29043 (mbergeron)
+- Support visual reviews on private and internal projects through PAT authentication. !29336
+- Nuget search: fix the prerelease filtering. !29482
+- Smartcard should be counted as 2fa. !29504
+
+### Changed (6 changes)
+
+- Default the :tasks_by_type_chart feature flag on. !28486
+- Remove Add Metric button from custom dashboards. !29036
+- Improve Geo Node save error messages. !29079
+- Cosmetic changes for Epic Health Status. !29234
+- Improve readability of Dependency List. !29593
+- Restrict prompt to check user account settings only to Gitlab.com. !29672
+
+### Added (15 changes)
+
+- Add LDAP user filter to group link API. !26202
+- Implement Shared Runner Minute Factors. !27792
+- Updates the package registry list UI which also includes adding pipeline information. !28426
+- Allow Admins to preview the payload for Seat Link requests. !28582
+- Add deployment frequency to Project level Value Stream Analytics summary. !28772
+- Add deployment frequency to Group Value Stream Analytics summary. !28776
+- Provide milestone burnup chart data for scope committed graph. !28899
+- Make Status Page MVC generally available. !28966
+- License Compliance - Add `order_by` filter. !28970
+- Add the selected compliance frameworks label to project home and listings. !29137
+- Generate smaller versions of Design Management design files. !29215
+- .com has a subscription expired banner. !29422
+- Allow changing item parent in epic tree via GraphQL. !29567
+- Add PyPI Packages Repository. !29702
+- Enable requirements for projects and basic actions (CRUD) for them.
+
+### Other (1 change)
+
+- Add health status counts to usage data. !28964
+
+
 ## 12.9.4 (2020-04-16)
 
 - No changes.

app/models/todo.rb

@@ -110,7 +110,7 @@ class Todo < ApplicationRecord
       base = where.not(state: new_state).except(:order)
       ids = base.pluck(:id)
 
-      base.update_all(state: new_state)
+      base.update_all(state: new_state, updated_at: Time.now)
 
       ids
     end

changelogs/unreleased/migrate-vulnerability-dismissals.yml

@@ -0,0 +1,5 @@
+---
+title: Migration of dismissals to vulnerabilities
+merge_request: 29711
+author:
+type: other

db/post_migrate/20200416111111_migrate_vulnerability_dismissals.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+class MigrateVulnerabilityDismissals < ActiveRecord::Migration[6.0]
+  include Gitlab::Database::MigrationHelpers
+
+  DOWNTIME = false
+
+  disable_ddl_transaction!
+
+  MIGRATION = 'UpdateVulnerabilitiesToDismissed'.freeze
+  BATCH_SIZE = 500
+  DELAY_INTERVAL = 2.minutes.to_i
+
+  class Vulnerability < ActiveRecord::Base
+    self.table_name = 'vulnerabilities'
+    self.inheritance_column = :_type_disabled
+
+    include ::EachBatch
+  end
+
+  def up
+    return unless Gitlab.ee?
+
+    Vulnerability.select('project_id').group(:project_id).each_batch(of: BATCH_SIZE, column: "project_id") do |project_batch, index|
+      batch_delay = (index - 1) * BATCH_SIZE * DELAY_INTERVAL
+
+      project_batch.each_with_index do |project, project_batch_index|
+        project_delay = project_batch_index * DELAY_INTERVAL
+        migrate_in(batch_delay + project_delay, MIGRATION, project[:project_id])
+      end
+    end
+  end
+
+  def down
+    # nothing to do
+  end
+end

db/structure.sql

@@ -13256,6 +13256,7 @@ COPY "schema_migrations" (version) FROM STDIN;
 20200415161021
 20200415161206
 20200415192656
+20200416111111
 20200416120128
 20200416120354
 \.

doc/administration/operations/puma.md

@@ -14,6 +14,12 @@ Most Rails applications requests normally include a proportion of I/O wait time.
 During I/O wait time MRI Ruby will release the GVL (Global VM Lock) to other threads.
 Multi-threaded Puma can therefore still serve more requests than a single process.
 
+## Configuring Puma to replace Unicorn
+
+If you are currently running Unicorn and would like to switch to Puma, server configuration
+will _not_ carry over automatically. For details on matching Unicorn configuration settings with
+the Puma equivalent, where applicable, see [Converting Unicorn settings to Puma](https://docs.gitlab.com/omnibus/settings/puma.html#converting-unicorn-settings-to-puma).
+
 ## Performance caveat when using Puma with Rugged
 
 For deployments where NFS is used to store Git repository, we allow GitLab to use

doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md

@@ -375,7 +375,7 @@ see if our latest code is running without errors.
 When we finish this edition, GitLab will start another build and show a **build running** badge. It
 is expected, after all we just configured GitLab CI/CD to do this for every push! But you may think
 "Why run build and tests for simple things like editing README.md?" and it is a good question.
-For changes that don't affect your application, you can add the keyword [`[ci skip]`](../../yaml/README.md#skipping-jobs)
+For changes that don't affect your application, you can add the keyword [`[ci skip]`](../../yaml/README.md#skip-pipeline)
 to commit message and the build related to that commit will be skipped.
 
 In the end, we finally got our pretty green build succeeded badge! By outputting the result on the

doc/install/requirements.md

@@ -180,28 +180,50 @@ For most instances we recommend using: (CPU cores * 1.5) + 1 = Unicorn workers.
 For example a node with 4 cores would have 7 Unicorn workers.
 
 For all machines that have 2GB and up we recommend a minimum of three Unicorn workers.
-If you have a 1GB machine we recommend to configure only two Unicorn workers to prevent excessive swapping.
+If you have a 1GB machine we recommend to configure only two Unicorn workers to prevent excessive
+swapping.
 
-As long as you have enough available CPU and memory capacity, it's okay to increase the number of Unicorn workers and this will usually help to reduce the response time of the applications and increase the ability to handle parallel requests.
+As long as you have enough available CPU and memory capacity, it's okay to increase the number of
+Unicorn workers and this will usually help to reduce the response time of the applications and
+increase the ability to handle parallel requests.
 
-To change the Unicorn workers when you have the Omnibus package (which defaults to the recommendation above) please see [the Unicorn settings in the Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/unicorn.html).
+To change the Unicorn workers when you have the Omnibus package (which defaults to the
+recommendation above) please see [the Unicorn settings in the Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/unicorn.html).
 
-## Puma Workers
+## Puma settings
 
-For most instances we recommend using: max(CPU cores * 0.9, 2) = Puma workers.
+The recommended settings for Puma are determined by the infrastructure on which it's running.
-For example a node with 4 cores would have 3 Puma workers.
+Omnibus GitLab defaults to the recommended Puma settings. Regardless of installation method, you can
+tune the Puma settings.
 
-For all machines that have 4GB and up we recommend a minimum of three Puma workers.
+If you're using Omnibus GitLab, see [Puma settings](https://docs.gitlab.com/omnibus/settings/puma.html)
-If you have a 2GB machine we recommend to configure only one Puma worker to prevent excessive swapping.
+for instructions on changing the Puma settings.
 
-By default each Puma worker runs with 4 threads. We do not recommend setting more,
+### Puma workers
-due to how [Ruby MRI multi-threading](https://en.wikipedia.org/wiki/Global_interpreter_lock) works.
 
-For cases when you have to use [Legacy Rugged code](../development/gitaly.md#legacy-rugged-code) it is recommended to set number of threads to 1.
+The recommended number of workers is calculated as the highest of the following:
 
-As long as you have enough available CPU and memory capacity, it's okay to increase the number of Puma workers and this will usually help to reduce the response time of the applications and increase the ability to handle parallel requests.
+- `2`
+- Number of CPU cores - 1
 
-To change the Puma workers when you have the Omnibus package (which defaults to the recommendation above) please see [the Puma settings in the Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/puma.html).
+For example a node with 4 cores should be configured with 3 Puma workers.
+
+You can increase the number of Puma workers, providing enough CPU and memory capacity is available.
+A higher number of Puma workers will usually help to reduce the response time of the application
+and increase the ability to handle parallel requests. You must perform testing to verify the
+optimal settings for your infrastructure.
+
+### Puma threads
+
+The recommended number of threads is dependent on several factors, including total memory, and use
+of [legacy Rugged code](../development/gitaly.md#legacy-rugged-code).
+
+- If the operating system has a maximum 2 GB of memory, the recommended number of threads is `1`.
+  A higher value will result in excess swapping, and decrease performance.
+- If legacy Rugged code is in use, the recommended number of threads is `1`.
+- In all other cases, the recommended number of threads is `4`. We do not recommend setting this
+higher, due to how [Ruby MRI multi-threading](https://en.wikipedia.org/wiki/Global_interpreter_lock)
+works.
 
 ## Redis and Sidekiq
 

doc/user/compliance/license_compliance/index.md

@@ -116,13 +116,14 @@ License Compliance can be configured using environment variables.
 
 | Environment variable  | Required | Description |
 |-----------------------|----------|-------------|
-| `MAVEN_CLI_OPTS`       | no | Additional arguments for the mvn executable. If not supplied, defaults to `-DskipTests`. |
+| `ADDITIONAL_CA_CERT_BUNDLE` | no | Bundle of trusted CA certificates (currently supported in Python projects). |
+| `GRADLE_CLI_OPTS` | no | Additional arguments for the gradle executable. If not supplied, defaults to `--exclude-task=test`. |
 | `LICENSE_FINDER_CLI_OPTS` | no | Additional arguments for the `license_finder` executable. For example, if your project has both Golang and Ruby code stored in different directories and you want to only scan the Ruby code, you can update your `.gitlab-ci-yml` template to specify which project directories to scan, like `LICENSE_FINDER_CLI_OPTS: '--debug --aggregate-paths=. ruby'`. |
 | `LM_JAVA_VERSION`      | no | Version of Java. If set to `11`, Maven and Gradle use Java 11 instead of Java 8. |
 | `LM_PYTHON_VERSION`    | no | Version of Python. If set to `3`, dependencies are installed using Python 3 instead of Python 2.7. |
-| `SETUP_CMD`            | no | Custom setup for the dependency installation. (experimental) |
+| `MAVEN_CLI_OPTS`       | no | Additional arguments for the mvn executable. If not supplied, defaults to `-DskipTests`. |
 | `PIP_INDEX_URL` | no | Base URL of Python Package Index (default: `https://pypi.org/simple/`). |
-| `ADDITIONAL_CA_CERT_BUNDLE` | no | Bundle of trusted CA certificates (currently supported in Python projects). |
+| `SETUP_CMD`            | no | Custom setup for the dependency installation (experimental). |
 
 ### Installing custom dependencies
 

doc/user/project/deploy_boards.md

@@ -130,6 +130,11 @@ spec:
 
 The annotations will be applied to the deployments, replica sets, and pods. By changing the number of replicas, like `kubectl scale --replicas=3 deploy APPLICATION_NAME -n ${KUBE_NAMESPACE}`, you can follow the instances' pods from the board.
 
+NOTE: **Note:**
+The YAML file is static. If you apply it using `kubectl apply`, you must
+manually provide the project and environment slugs, or create a script to
+replace the variables in the YAML before applying.
+
 ## Canary Deployments
 
 A popular CI strategy, where a small portion of the fleet is updated to the new

doc/user/project/pages/custom_domains_ssl_tls_certification/index.md

@@ -61,8 +61,8 @@ according to the type of domain you want to use with your Pages site:
 - [For both](#for-both-root-and-subdomains).
 
 NOTE: **Note:**
-IPv6 is not currently configured for Pages on GitLab.com, but
+You can [configure IPv6 on self-managed instances].(../../../../administration/pages/index.md#advanced-configuration),
-you can [configure IPv6 on self-managed instances].(../../../../administration/pages/index.md#advanced-configuration).
+but IPv6 is not currently configured for Pages on GitLab.com.
 Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/214718) for details.
 
 ##### For root domains

doc/user/project/repository/index.md

@@ -162,7 +162,7 @@ Via command line, you can commit multiple times before pushing.
   you will trigger a pipeline per push, not per commit.
   - **Skip pipelines:**
   You can add to you commit message the keyword
-  [`[ci skip]`](../../../ci/yaml/README.md#skipping-jobs)
+  [`[ci skip]`](../../../ci/yaml/README.md#skip-pipeline)
   and GitLab CI/CD will skip that pipeline.
   - **Cross-link issues and merge requests:**
   [Cross-linking](../issues/crosslinking_issues.md#from-commit-messages)

doc/user/project/settings/import_export.md

@@ -25,6 +25,8 @@ To set up a project import/export:
 
 Note the following:
 
+- Imports from a newer version of GitLab are not supported.
+  The Importing GitLab version must be greater than or equal to the Exporting GitLab version.
 - Imports will fail unless the import and export GitLab instances are
   compatible as described in the [Version history](#version-history).
 - Exports are stored in a temporary [shared directory](../../../development/shared_files.md)

lib/gitlab/background_migration/update_vulnerabilities_to_dismissed.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module Gitlab
+  module BackgroundMigration
+    # rubocop: disable Style/Documentation
+    class UpdateVulnerabilitiesToDismissed
+      def perform(project_id)
+      end
+    end
+  end
+end
+
+Gitlab::BackgroundMigration::UpdateVulnerabilitiesToDismissed.prepend_if_ee('EE::Gitlab::BackgroundMigration::UpdateVulnerabilitiesToDismissed')

qa/qa/specs/features/browser_ui/2_plan/issue/check_mentions_for_xss_spec.rb

@@ -3,33 +3,42 @@
 module QA
   context 'Plan', :reliable do
     describe 'check xss occurence in @mentions in issues', :requires_admin do
-      it 'mentions a user in a comment' do
+      let(:user) do
+        Resource::User.fabricate_via_api! do |user|
+          user.name = "eve <img src=x onerror=alert(2)&lt;img src=x onerror=alert(1)&gt;"
+          user.password = "test1234"
+        end
+      end
+
+      let(:project) do
+        Resource::Project.fabricate_via_api! do |project|
+          project.name = 'xss-test-for-mentions-project'
+          project.add_member(user)
+        end
+      end
+
+      let(:issue) do
+        Resource::Issue.fabricate_via_api! do |issue|
+          issue.project = project
+        end
+      end
+
+      before do
         QA::Runtime::Env.personal_access_token = QA::Runtime::Env.admin_personal_access_token
 
         unless QA::Runtime::Env.personal_access_token
           Flow::Login.sign_in_as_admin
         end
 
-        user = Resource::User.fabricate_via_api! do |user|
-          user.name = "eve <img src=x onerror=alert(2)&lt;img src=x onerror=alert(1)&gt;"
-          user.password = "test1234"
-        end
-
         QA::Runtime::Env.personal_access_token = nil
 
         Page::Main::Menu.perform(&:sign_out) if Page::Main::Menu.perform { |p| p.has_personal_area?(wait: 0) }
 
         Flow::Login.sign_in
-
-        project = Resource::Project.fabricate_via_api! do |project|
-          project.name = 'xss-test-for-mentions-project'
       end
 
-        Flow::Project.add_member(project: project, username: user.username)
+      it 'mentions a user in a comment' do
-
+        issue.visit!
-        Resource::Issue.fabricate_via_api! do |issue|
-          issue.project = project
-        end.visit!
 
         Page::Project::Issue::Show.perform do |show|
           show.select_all_activities_filter

spec/models/todo_spec.rb

@@ -389,5 +389,17 @@ describe Todo do
 
       expect(described_class.update_state(:pending)).to be_empty
     end
+
+    it 'updates updated_at' do
+      create(:todo, :pending)
+
+      Timecop.freeze(1.day.from_now) do
+        expected_update_date = Time.now.utc
+
+        ids = described_class.update_state(:done)
+
+        expect(Todo.where(id: ids).map(&:updated_at)).to all(be_like_time(expected_update_date))
+      end
+    end
   end
 end