kotovalexarian-likes-gitlab
/
gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Add latest changes from gitlab-org/gitlab@master

This commit is contained in:
GitLab Bot 2021-09-30 18:11:31 +00:00
parent eaec42f9e3
commit c753fd0bf4
198 changed files with 4807 additions and 2948 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.eslintrc.yml
View File

@ -100,6 +100,7 @@ overrides:
- 'scripts/**/*'
- '*.config.js'
- '*.config.*.js'
- storybook/config/*.js
rules:
'@gitlab/require-i18n-strings': off
import/no-extraneous-dependencies: off

5
.gitignore vendored
View File

@ -40,10 +40,7 @@ eslint-report.html
/config/initializers/smtp_settings.rb
/config/initializers/relative_url.rb
/config/resque.yml
/config/redis.cache.yml
/config/redis.queues.yml
/config/redis.shared_state.yml
/config/redis.trace_chunks.yml
/config/redis.*.yml
/config/unicorn.rb
/config/puma.rb
/config/secrets.yml

8
.gitlab/ci/frontend.gitlab-ci.yml
View File

@ -366,6 +366,11 @@ compile-storybook:
extends:
- .compile-storybook-base
- .frontend:rules:default-frontend-jobs
needs:
- job: "rspec frontend_fixture"
- job: "rspec-ee frontend_fixture"
optional: true
- job: "graphql-schema-dump"
artifacts:
name: storybook
expire_in: 31d
@ -378,3 +383,6 @@ compile-storybook as-if-foss:
- .compile-storybook-base
- .as-if-foss
- .frontend:rules:default-frontend-jobs-as-if-foss
needs:
- job: "graphql-schema-dump"
- job: "rspec frontend_fixture as-if-foss"

1
.stylelintrc
View File

@ -13,6 +13,7 @@
"./scripts/frontend/stylelint/stylelint-utility-classes.js",
],
"rules":{
"at-rule-disallowed-list": ["extend"],
"max-nesting-depth": [
3,
{

100
CHANGELOG.md
View File

@ -2,6 +2,40 @@
documentation](doc/development/changelog.md) for instructions on adding your own
entry.
## 14.3.1 (2021-09-30)
### Security (29 changes)
- [Fix permissions check on project members import](gitlab-org/security/gitlab@63ba9ad2a1067eb74df493e273707bb64a13a197) ([merge request](gitlab-org/security/gitlab!1858))
- [Require password param for 2FA changes](gitlab-org/security/gitlab@f246cfbd15344ba74a0182276bf63f0b5f1a4a31) ([merge request](gitlab-org/security/gitlab!1813))
- [Respect disabled import sources when initiating import via API](gitlab-org/security/gitlab@046e964b0151fc8c58063281a39af063ffb678bd) ([merge request](gitlab-org/security/gitlab!1846))
- [Return 404 if model id wasn't passed to UploadsController](gitlab-org/security/gitlab@747e6f0e4aec39462f296fd56b37df1c255d29cb) ([merge request](gitlab-org/security/gitlab!1843))
- [Scrub artifacts signed URL in SendEntry logs](gitlab-org/security/gitlab@f6c57892ddc9518efaace1021346b42b4c805a1c) ([merge request](gitlab-org/security/gitlab!1840))
- [Prevent double-impersonation and impersonation breakout](gitlab-org/security/gitlab@615d418f9315ca3b3619689c47201f618cf6bde9) ([merge request](gitlab-org/security/gitlab!1834))
- [Clear session access tokens when starting/stopping impersonation](gitlab-org/security/gitlab@62c2e0d3ed73f2d7ded90d04fe232ff6ae2f6136) ([merge request](gitlab-org/security/gitlab!1831))
- [Prevent users from bypassing 2FA on certain pages](gitlab-org/security/gitlab@0b41838b36da09a9230de4d8449040a701464de7) ([merge request](gitlab-org/security/gitlab!1827))
- [Use validated URL when sending request to Gitea Importer](gitlab-org/security/gitlab@26731d762f6503fe1b8b509be11c56e77601a552) ([merge request](gitlab-org/security/gitlab!1822))
- [Fix XSS in Jira link](gitlab-org/security/gitlab@d41060acb2aa151119042db9162a102d4e2c15ab) ([merge request](gitlab-org/security/gitlab!1819)) **GitLab Enterprise Edition**
- [Fix fogbugz importer DNS Rebind SSRF](gitlab-org/security/gitlab@cc13d57c66cc65e6f920bdeab57b9fdb9d6baac1) ([merge request](gitlab-org/security/gitlab!1814))
- [Remove related project access tokens when a project is deleted](gitlab-org/security/gitlab@d32c0d57d5b39601034c4c4ae983ea80c05db429) ([merge request](gitlab-org/security/gitlab!1810))
- [Require group admin access to list pending invites](gitlab-org/security/gitlab@911bb0cb78e00934c491af59729fa84fffae7676) ([merge request](gitlab-org/security/gitlab!1793))
- [Do not export and import repository_size_limit](gitlab-org/security/gitlab@0f3feca459895fc6665f8b0dfc16d4dcd7112944) ([merge request](gitlab-org/security/gitlab!1770))
- [Escapes MR approval rule names correctly](gitlab-org/security/gitlab@4fcd97230bbc31780fe14d75694bb6433d57e677) ([merge request](gitlab-org/security/gitlab!1807))
- [Filter shared groups autocomplete by permitted](gitlab-org/security/gitlab@b5144abb0516af61686402c2ad720967d11cb03c) ([merge request](gitlab-org/security/gitlab!1804)) **GitLab Enterprise Edition**
- [Require access token for git when 2fa is required](gitlab-org/security/gitlab@ea22f67b47bf0d6c801f2bf6c9672c0ea5afd30c) ([merge request](gitlab-org/security/gitlab!1794))
- [Prohibit anonymous access for specific user API endpoint](gitlab-org/security/gitlab@c52890997ad574812ae4da968f2f6ecfd9f7ff59) ([merge request](gitlab-org/security/gitlab!1792))
- [Disable exporting pipeline triggers on project export](gitlab-org/security/gitlab@f7f18fbdd8e81a9b3e0650250316c7bb17ac1956) ([merge request](gitlab-org/security/gitlab!1791))
- [Add pagination to dependencies API](gitlab-org/security/gitlab@203328889059564ba6085663b21355149c01e501) ([merge request](gitlab-org/security/gitlab!1726)) **GitLab Enterprise Edition**
- [Do not allow status checks to exist with external protected branches](gitlab-org/security/gitlab@327d8080e7e7b0bc77b7933f8026ec0cf1abd99a) ([merge request](gitlab-org/security/gitlab!1788)) **GitLab Enterprise Edition**
- [Permission check issuable template API data](gitlab-org/security/gitlab@de7851c2ab58c31df49c8a406ed0c3f3ad779e26) ([merge request](gitlab-org/security/gitlab!1785)) **GitLab Enterprise Edition**
- [Apply account locking to password reset page](gitlab-org/security/gitlab@050dfa71191ffaea77a4a18e0dea1f3336f40db5) ([merge request](gitlab-org/security/gitlab!1782))
- [Enforce configured scopes for Oauth applications](gitlab-org/security/gitlab@ce83bb14b5a4521f889086a439f1628041843589) ([merge request](gitlab-org/security/gitlab!1779))
- [Verify state before using errors from OAuth2 OmniAuth providers](gitlab-org/security/gitlab@dcc2cad6c03255ac70f29ed9c0f6c8bc11ac1018) ([merge request](gitlab-org/security/gitlab!1776))
- [Prevent moving epic issues to different group hierarchy](gitlab-org/security/gitlab@167601717f2ad46fee2320af6ac49674026501be) ([merge request](gitlab-org/security/gitlab!1772)) **GitLab Enterprise Edition**
- [Fix GFM autocomplete xss](gitlab-org/security/gitlab@8816ab6af1d1aa752f22da7850d4d1c983f2d43a) ([merge request](gitlab-org/security/gitlab!1767))
- [Prevent showing not allowed subgroup epics](gitlab-org/security/gitlab@b841c78c47b6a56b618186720bffc26922807356) ([merge request](gitlab-org/security/gitlab!1764)) **GitLab Enterprise Edition**
- [Fix denial-of-service attack in Markdown parser](gitlab-org/security/gitlab@5e5973b5c28862381729408ba4df650c3d4f7ce0) ([merge request](gitlab-org/security/gitlab!1730))
## 14.3.0 (2021-09-21)
### Added (111 changes)
@ -498,6 +532,39 @@ entry.
- [Remove the FF ci_reset_bridge_with_subsequent_jobs](gitlab-org/gitlab@a4a75095b9b0250d0b1bdadea90c8a4cd24449b2) ([merge request](gitlab-org/gitlab!68295))
- [Removes ci_same_stage_job_needs ff](gitlab-org/gitlab@5e509cf7aa90041a541b19dda563120a359f0bf9) ([merge request](gitlab-org/gitlab!68041))
## 14.2.5 (2021-09-30)
### Security (28 changes)
- [Require password param for 2FA changes](gitlab-org/security/gitlab@5693760a3edf82774a4e19b9bb561be87316be54) ([merge request](gitlab-org/security/gitlab!1815))
- [Fix permissions check on project members import](gitlab-org/security/gitlab@f9b4200427833e370638aa63851d6801a40c404c) ([merge request](gitlab-org/security/gitlab!1859))
- [Respect disabled import sources when initiating import via API](gitlab-org/security/gitlab@3c9af055ece281fcaab0b8dcc277e0ce9133ad31) ([merge request](gitlab-org/security/gitlab!1847))
- [Return 404 if model id wasn't passed to UploadsController](gitlab-org/security/gitlab@dd4616362040d2b812d69ff2ecf78e70ff4b9ae3) ([merge request](gitlab-org/security/gitlab!1844))
- [Scrub artifacts signed URL in SendEntry logs](gitlab-org/security/gitlab@41d275bb73943ba6f970d6678b49c9336194af35) ([merge request](gitlab-org/security/gitlab!1841))
- [Prevent double-impersonation and impersonation breakout](gitlab-org/security/gitlab@c0ab498adda057f4d87969d74c32a3ea95df297c) ([merge request](gitlab-org/security/gitlab!1835))
- [Clear session access tokens when starting/stopping impersonation](gitlab-org/security/gitlab@fd39d88b348d525818820d2496afe08612420023) ([merge request](gitlab-org/security/gitlab!1832))
- [Use validated URL when sending request to Gitea Importer](gitlab-org/security/gitlab@328e3c726c693b32666e0fb32eda0b7a6f22d8ad) ([merge request](gitlab-org/security/gitlab!1821))
- [Fix XSS in Jira link](gitlab-org/security/gitlab@868d8b9c4a1e9e9019a7ff51da11f75051e452c3) ([merge request](gitlab-org/security/gitlab!1817)) **GitLab Enterprise Edition**
- [Fix fogbugz importer DNS Rebind SSRF](gitlab-org/security/gitlab@4f4b5a15a3508084f921442b3a7f42ba0448f1bb) ([merge request](gitlab-org/security/gitlab!1681))
- [Remove related project access tokens when a project is deleted](gitlab-org/security/gitlab@282e81198f80f1fda912da5bc6f671d778b19ca9) ([merge request](gitlab-org/security/gitlab!1811))
- [Require group admin access to list pending invites](gitlab-org/security/gitlab@1ce85345787025222c915fe5fa314bad8994b6ba) ([merge request](gitlab-org/security/gitlab!1720))
- [Do not export and import repository_size_limit](gitlab-org/security/gitlab@359f14e41dfc355a13041cdf1dbcd082c254200c) ([merge request](gitlab-org/security/gitlab!1769))
- [Escapes MR approval rule names correctly](gitlab-org/security/gitlab@d84739982599197ff337d69d818634544270e142) ([merge request](gitlab-org/security/gitlab!1808))
- [Filter shared groups autocomplete by permitted](gitlab-org/security/gitlab@3a2b4c7ff1eb2ba3e84840ba2800c13d6491d726) ([merge request](gitlab-org/security/gitlab!1805)) **GitLab Enterprise Edition**
- [Require access token for git when 2fa is required](gitlab-org/security/gitlab@deb4e7e5f941c82450d382c1b85f6325e367394f) ([merge request](gitlab-org/security/gitlab!1795))
- [Disable exporting pipeline triggers on project export](gitlab-org/security/gitlab@417761bb2f67f03bfe803163bad97da7b9fa088b) ([merge request](gitlab-org/security/gitlab!1789))
- [Add pagination to dependencies API](gitlab-org/security/gitlab@2f84755ba54580df126054a561d8cc4731f936d3) ([merge request](gitlab-org/security/gitlab!1724)) **GitLab Enterprise Edition**
- [Permission check issuable template API data](gitlab-org/security/gitlab@a90614e2efc813ca5f13a9aa9b51f13f0e8934aa) ([merge request](gitlab-org/security/gitlab!1786)) **GitLab Enterprise Edition**
- [Apply account locking to password reset page](gitlab-org/security/gitlab@6bbd77c0748e59eacff51edb6264d6099ee14a38) ([merge request](gitlab-org/security/gitlab!1783))
- [Enforce configured scopes for Oauth applications](gitlab-org/security/gitlab@a9f44bb19cbfc460cd05627a80ef17c39cdde86b) ([merge request](gitlab-org/security/gitlab!1780))
- [Verify state before using errors from OAuth2 OmniAuth providers](gitlab-org/security/gitlab@6f70292d0fa3efbe99c44748a463df189830cc35) ([merge request](gitlab-org/security/gitlab!1777))
- [Prevent moving epic issues to different group hierarchy](gitlab-org/security/gitlab@979d40003794014d5930709a257e9a5c75df10e6) ([merge request](gitlab-org/security/gitlab!1773)) **GitLab Enterprise Edition**
- [Prevent showing not allowed subgroup epics](gitlab-org/security/gitlab@2f72e4062f6cd7256ffff31172b00c012a5910e1) ([merge request](gitlab-org/security/gitlab!1765)) **GitLab Enterprise Edition**
- [Do not allow status checks to exist with external protected branches](gitlab-org/security/gitlab@dd08837d054c574f94f80e806cc7b49de342cc57) ([merge request](gitlab-org/security/gitlab!1762)) **GitLab Enterprise Edition**
- [Fix GFM autocomplete xss](gitlab-org/security/gitlab@5afba618ef89fdce544f498a30e7366e3f6cb788) ([merge request](gitlab-org/security/gitlab!1747))
- [Prohibit anonymous access for specific user API endpoint](gitlab-org/security/gitlab@a813bd8a8f07ffa0477efd3a3936b436e5ec6b17) ([merge request](gitlab-org/security/gitlab!1736))
- [Fix denial-of-service attack in Markdown parser](gitlab-org/security/gitlab@f618ad9c104882ac5f707b162e8119805252019e) ([merge request](gitlab-org/security/gitlab!1729))
## 14.2.4 (2021-09-17)
### Fixed (2 changes)
@ -1089,6 +1156,39 @@ entry.
- [Add helpful text to URL group validation and limit text](gitlab-org/gitlab@59a5a6266cb0d5434596170ffa36e4e74b8d2c2c) ([merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65369)) **GitLab Enterprise Edition**
- [Refactor external storage admin area configuration UI and docs](gitlab-org/gitlab@497ba4fc8f4ec1d234c9f5f1ec5c69712b8c7cb3) ([merge request](gitlab-org/gitlab!66219))
## 14.1.7 (2021-09-30)
### Security (28 changes)
- [Require password param for 2FA changes](gitlab-org/security/gitlab@4e16401a77264ef3127f0bb314fa0abab11216c7) ([merge request](gitlab-org/security/gitlab!1816))
- [Fix permissions check on project members import](gitlab-org/security/gitlab@be54b3f4890fa89d58cb02be79c65025f606bd6c) ([merge request](gitlab-org/security/gitlab!1860))
- [Respect disabled import sources when initiating import via API](gitlab-org/security/gitlab@b76b6229c93447954efc5719e6dd61eb601afdc4) ([merge request](gitlab-org/security/gitlab!1848))
- [Return 404 if model id wasn't passed to UploadsController](gitlab-org/security/gitlab@8ab1cfaafba21a9e90ef41677993af2afdcd920a) ([merge request](gitlab-org/security/gitlab!1845))
- [Scrub artifacts signed URL in SendEntry logs](gitlab-org/security/gitlab@5bae93b2f085d5dac58e411bffb5ca518fe7df98) ([merge request](gitlab-org/security/gitlab!1842))
- [Prevent double-impersonation and impersonation breakout](gitlab-org/security/gitlab@7c82d0f2a1dbfb0a23d6a5cdaee448307ffc6972) ([merge request](gitlab-org/security/gitlab!1836))
- [Clear session access tokens when starting/stopping impersonation](gitlab-org/security/gitlab@a4d529eb7c8eeb3e9c42ae3d81514d79a905cdb7) ([merge request](gitlab-org/security/gitlab!1833))
- [Use validated URL when sending request to Gitea Importer](gitlab-org/security/gitlab@b30536e6c9aa969c76bcd167f00db5a0e07ace7a) ([merge request](gitlab-org/security/gitlab!1820))
- [Fix XSS in Jira link](gitlab-org/security/gitlab@9ace10c46744ee220c649d2da0eeb3e99216ee7d) ([merge request](gitlab-org/security/gitlab!1625)) **GitLab Enterprise Edition**
- [Fix fogbugz importer DNS Rebind SSRF](gitlab-org/security/gitlab@9d7107665d6ed931ef4b2feeb0287bc71b89232c) ([merge request](gitlab-org/security/gitlab!1682))
- [Remove related project access tokens when a project is deleted](gitlab-org/security/gitlab@b86096865949f07f6a2020603959117d9c84877b) ([merge request](gitlab-org/security/gitlab!1812))
- [Require group admin access to list pending invites](gitlab-org/security/gitlab@404b344edd61b2f13c3498cd545c2b40165ee536) ([merge request](gitlab-org/security/gitlab!1721))
- [Do not export and import repository_size_limit](gitlab-org/security/gitlab@56f563980f944f1a5e3935ad82070e6719cd5a0c) ([merge request](gitlab-org/security/gitlab!1768))
- [Escapes MR approval rule names correctly](gitlab-org/security/gitlab@ea64f981ce70a0e1e6ee58e64a6007e82f48e071) ([merge request](gitlab-org/security/gitlab!1809))
- [Filter shared groups autocomplete by permitted](gitlab-org/security/gitlab@59999ab27cba402589b27d204cf29678100e948b) ([merge request](gitlab-org/security/gitlab!1806)) **GitLab Enterprise Edition**
- [Require access token for git when 2fa is required](gitlab-org/security/gitlab@6a4a75efd7685a69ffa7cc4c027c7058013cca45) ([merge request](gitlab-org/security/gitlab!1796))
- [Disable exporting pipeline triggers on project export](gitlab-org/security/gitlab@8a8c78ed054def210013a849195939d7888fcf65) ([merge request](gitlab-org/security/gitlab!1790))
- [Add pagination to dependencies API](gitlab-org/security/gitlab@2a963ad670c60d1f3078fdf446ea755c5862fa26) ([merge request](gitlab-org/security/gitlab!1725)) **GitLab Enterprise Edition**
- [Permission check issuable template API data](gitlab-org/security/gitlab@9d95d13bc714e46b5e3697288c4b398cb5aee88b) ([merge request](gitlab-org/security/gitlab!1787)) **GitLab Enterprise Edition**
- [Apply account locking to password reset page](gitlab-org/security/gitlab@47ee79b1983de886f5ebe04b2975c2e37aa938ce) ([merge request](gitlab-org/security/gitlab!1784))
- [Enforce configured scopes for Oauth applications](gitlab-org/security/gitlab@acf2d894c91aa7fb72ea32b10e50e94441885399) ([merge request](gitlab-org/security/gitlab!1781))
- [Verify state before using errors from OAuth2 OmniAuth providers](gitlab-org/security/gitlab@20073576508aa239e52d8ff911c1dfd3df8af670) ([merge request](gitlab-org/security/gitlab!1778))
- [Prevent moving epic issues to different group hierarchy](gitlab-org/security/gitlab@93c6ec69b7bc6c9124a2a5350cebebb57f63a28f) ([merge request](gitlab-org/security/gitlab!1774)) **GitLab Enterprise Edition**
- [Prevent showing not allowed subgroup epics](gitlab-org/security/gitlab@72a11e72425a033f3464d6ff12b4d06e12ec9faf) ([merge request](gitlab-org/security/gitlab!1766)) **GitLab Enterprise Edition**
- [Do not allow status checks to exist with external protected branches](gitlab-org/security/gitlab@8f96c013ccbbe9c52b3f03fb0d247debb1b157a8) ([merge request](gitlab-org/security/gitlab!1763)) **GitLab Enterprise Edition**
- [Fix GFM autocomplete xss](gitlab-org/security/gitlab@fd92dabddff5ae5d67a98aef5d858438520a2f06) ([merge request](gitlab-org/security/gitlab!1748))
- [Prohibit anonymous access for specific user API endpoint](gitlab-org/security/gitlab@2e8a386430309a931dbbd47fba7540a53399ad64) ([merge request](gitlab-org/security/gitlab!1737))
- [Fix denial-of-service attack in Markdown parser](gitlab-org/security/gitlab@5b6ed5212f880e2397dbea9ffc74cf0a35bd4411) ([merge request](gitlab-org/security/gitlab!1728))
## 14.1.6 (2021-09-27)
### Fixed (1 change)

98
app/assets/javascripts/authentication/two_factor_auth/components/manage_two_factor_form.vue Normal file
View File

@ -0,0 +1,98 @@
<script>
import { GlFormInput, GlFormGroup, GlButton, GlForm } from '@gitlab/ui';
import csrf from '~/lib/utils/csrf';
import { __ } from '~/locale';
export const i18n = {
currentPassword: __('Current password'),
confirmWebAuthn: __(
'Are you sure? This will invalidate your registered applications and U2F / WebAuthn devices.',
),
confirm: __('Are you sure? This will invalidate your registered applications and U2F devices.'),
disableTwoFactor: __('Disable two-factor authentication'),
regenerateRecoveryCodes: __('Regenerate recovery codes'),
};
export default {
name: 'ManageTwoFactorForm',
i18n,
components: {
GlForm,
GlFormInput,
GlFormGroup,
GlButton,
},
inject: [
'webauthnEnabled',
'profileTwoFactorAuthPath',
'profileTwoFactorAuthMethod',
'codesProfileTwoFactorAuthPath',
'codesProfileTwoFactorAuthMethod',
],
data() {
return {
method: '',
action: '#',
};
},
computed: {
confirmText() {
if (this.webauthnEnabled) {
return i18n.confirmWebAuthn;
}
return i18n.confirm;
},
},
methods: {
handleFormSubmit(event) {
this.method = event.submitter.dataset.formMethod;
this.action = event.submitter.dataset.formAction;
},
},
csrf,
};
</script>
<template>
<gl-form
class="gl-display-inline-block"
method="post"
:action="action"
@submit="handleFormSubmit($event)"
>
<input type="hidden" name="_method" data-testid="test-2fa-method-field" :value="method" />
<input :value="$options.csrf.token" type="hidden" name="authenticity_token" />
<gl-form-group :label="$options.i18n.currentPassword" label-for="current-password">
<gl-form-input
id="current-password"
type="password"
name="current_password"
required
data-qa-selector="current_password_field"
/>
</gl-form-group>
<gl-button
type="submit"
class="btn-danger gl-mr-3 gl-display-inline-block"
data-testid="test-2fa-disable-button"
variant="danger"
:data-confirm="confirmText"
:data-form-action="profileTwoFactorAuthPath"
:data-form-method="profileTwoFactorAuthMethod"
>
{{ $options.i18n.disableTwoFactor }}
</gl-button>
<gl-button
type="submit"
class="gl-display-inline-block"
data-testid="test-2fa-regenerate-codes-button"
:data-form-action="codesProfileTwoFactorAuthPath"
:data-form-method="codesProfileTwoFactorAuthMethod"
>
{{ $options.i18n.regenerateRecoveryCodes }}
</gl-button>
</gl-form>
</template>

31
app/assets/javascripts/authentication/two_factor_auth/index.js
View File

@ -1,8 +1,39 @@
import Vue from 'vue';
import { updateHistory, removeParams } from '~/lib/utils/url_utility';
import ManageTwoFactorForm from './components/manage_two_factor_form.vue';
import RecoveryCodes from './components/recovery_codes.vue';
import { SUCCESS_QUERY_PARAM } from './constants';
export const initManageTwoFactorForm = () => {
const el = document.querySelector('.js-manage-two-factor-form');
if (!el) {
return false;
}
const {
webauthnEnabled = false,
profileTwoFactorAuthPath = '',
profileTwoFactorAuthMethod = '',
codesProfileTwoFactorAuthPath = '',
codesProfileTwoFactorAuthMethod = '',
} = el.dataset;
return new Vue({
el,
provide: {
webauthnEnabled,
profileTwoFactorAuthPath,
profileTwoFactorAuthMethod,
codesProfileTwoFactorAuthPath,
codesProfileTwoFactorAuthMethod,
},
render(createElement) {
return createElement(ManageTwoFactorForm);
},
});
};
export const initRecoveryCodes = () => {
const el = document.querySelector('.js-2fa-recovery-codes');

33
app/assets/javascripts/gfm_auto_complete.js
View File

@ -1,6 +1,6 @@
import $ from 'jquery';
import '~/lib/utils/jquery_at_who';
import { escape, sortBy, template } from 'lodash';
import { escape as lodashEscape, sortBy, template } from 'lodash';
import * as Emoji from '~/emoji';
import axios from '~/lib/utils/axios_utils';
import { s__, __, sprintf } from '~/locale';
@ -11,8 +11,21 @@ import { spriteIcon } from './lib/utils/common_utils';
import { parsePikadayDate } from './lib/utils/datetime_utility';
import glRegexp from './lib/utils/regexp';
function sanitize(str) {
return str.replace(/<(?:.|\n)*?>/gm, '');
/**
* Escapes user input before we pass it to at.js, which
* renders it as HTML in the autocomplete dropdown.
*
* at.js allows you to reference data using `${}` syntax
* (e.g. ${search}) which it replaces with the actual data
* before rendering it in the autocomplete dropdown.
* To prevent user input from executing this `${}` syntax,
* we also need to escape the $ character.
*
* @param string user input
* @return {string} escaped user input
*/
function escape(string) {
return lodashEscape(string).replace(/\$/g, '&dollar;');
}
function createMemberSearchString(member) {
@ -44,8 +57,8 @@ export function membersBeforeSave(members) {
return {
username: member.username,
avatarTag: autoCompleteAvatar.length === 1 ? txtAvatar : imgAvatar,
title: sanitize(title),
search: sanitize(createMemberSearchString(member)),
title,
search: createMemberSearchString(member),
icon: avatarIcon,
availability: member?.availability,
};
@ -366,7 +379,7 @@ class GfmAutoComplete {
}
return {
id: i.iid,
title: sanitize(i.title),
title: i.title,
reference: i.reference,
search: `${i.iid} ${i.title}`,
};
@ -404,7 +417,7 @@ class GfmAutoComplete {
return {
id: m.iid,
title: sanitize(m.title),
title: m.title,
search: m.title,
expired,
dueDate,
@ -456,7 +469,7 @@ class GfmAutoComplete {
}
return {
id: m.iid,
title: sanitize(m.title),
title: m.title,
reference: m.reference,
search: `${m.iid} ${m.title}`,
};
@ -492,7 +505,7 @@ class GfmAutoComplete {
beforeSave(merges) {
if (GfmAutoComplete.isLoading(merges)) return merges;
return $.map(merges, (m) => ({
title: sanitize(m.title),
title: m.title,
color: m.color,
search: m.title,
set: m.set,
@ -586,7 +599,7 @@ class GfmAutoComplete {
}
return {
id: m.id,
title: sanitize(m.title),
title: m.title,
search: `${m.id} ${m.title}`,
};
});

8
app/assets/javascripts/lib/utils/regexp.js
View File

@ -1,6 +1,5 @@
/**
* Regexp utility for the convenience of working with regular expressions.
*
*/
// Inspired by https://github.com/mishoo/UglifyJS/blob/2bc1d02363db3798d5df41fb5059a19edca9b7eb/lib/parse-js.js#L203
@ -8,4 +7,9 @@
const unicodeLetters =
'\\u0041-\\u005A\\u0061-\\u007A\\u00AA\\u00B5\\u00BA\\u00C0-\\u00D6\\u00D8-\\u00F6\\u00F8-\\u02C1\\u02C6-\\u02D1\\u02E0-\\u02E4\\u02EC\\u02EE\\u0370-\\u0374\\u0376\\u0377\\u037A-\\u037D\\u0386\\u0388-\\u038A\\u038C\\u038E-\\u03A1\\u03A3-\\u03F5\\u03F7-\\u0481\\u048A-\\u0527\\u0531-\\u0556\\u0559\\u0561-\\u0587\\u05D0-\\u05EA\\u05F0-\\u05F2\\u0620-\\u064A\\u066E\\u066F\\u0671-\\u06D3\\u06D5\\u06E5\\u06E6\\u06EE\\u06EF\\u06FA-\\u06FC\\u06FF\\u0710\\u0712-\\u072F\\u074D-\\u07A5\\u07B1\\u07CA-\\u07EA\\u07F4\\u07F5\\u07FA\\u0800-\\u0815\\u081A\\u0824\\u0828\\u0840-\\u0858\\u08A0\\u08A2-\\u08AC\\u0904-\\u0939\\u093D\\u0950\\u0958-\\u0961\\u0971-\\u0977\\u0979-\\u097F\\u0985-\\u098C\\u098F\\u0990\\u0993-\\u09A8\\u09AA-\\u09B0\\u09B2\\u09B6-\\u09B9\\u09BD\\u09CE\\u09DC\\u09DD\\u09DF-\\u09E1\\u09F0\\u09F1\\u0A05-\\u0A0A\\u0A0F\\u0A10\\u0A13-\\u0A28\\u0A2A-\\u0A30\\u0A32\\u0A33\\u0A35\\u0A36\\u0A38\\u0A39\\u0A59-\\u0A5C\\u0A5E\\u0A72-\\u0A74\\u0A85-\\u0A8D\\u0A8F-\\u0A91\\u0A93-\\u0AA8\\u0AAA-\\u0AB0\\u0AB2\\u0AB3\\u0AB5-\\u0AB9\\u0ABD\\u0AD0\\u0AE0\\u0AE1\\u0B05-\\u0B0C\\u0B0F\\u0B10\\u0B13-\\u0B28\\u0B2A-\\u0B30\\u0B32\\u0B33\\u0B35-\\u0B39\\u0B3D\\u0B5C\\u0B5D\\u0B5F-\\u0B61\\u0B71\\u0B83\\u0B85-\\u0B8A\\u0B8E-\\u0B90\\u0B92-\\u0B95\\u0B99\\u0B9A\\u0B9C\\u0B9E\\u0B9F\\u0BA3\\u0BA4\\u0BA8-\\u0BAA\\u0BAE-\\u0BB9\\u0BD0\\u0C05-\\u0C0C\\u0C0E-\\u0C10\\u0C12-\\u0C28\\u0C2A-\\u0C33\\u0C35-\\u0C39\\u0C3D\\u0C58\\u0C59\\u0C60\\u0C61\\u0C85-\\u0C8C\\u0C8E-\\u0C90\\u0C92-\\u0CA8\\u0CAA-\\u0CB3\\u0CB5-\\u0CB9\\u0CBD\\u0CDE\\u0CE0\\u0CE1\\u0CF1\\u0CF2\\u0D05-\\u0D0C\\u0D0E-\\u0D10\\u0D12-\\u0D3A\\u0D3D\\u0D4E\\u0D60\\u0D61\\u0D7A-\\u0D7F\\u0D85-\\u0D96\\u0D9A-\\u0DB1\\u0DB3-\\u0DBB\\u0DBD\\u0DC0-\\u0DC6\\u0E01-\\u0E30\\u0E32\\u0E33\\u0E40-\\u0E46\\u0E81\\u0E82\\u0E84\\u0E87\\u0E88\\u0E8A\\u0E8D\\u0E94-\\u0E97\\u0E99-\\u0E9F\\u0EA1-\\u0EA3\\u0EA5\\u0EA7\\u0EAA\\u0EAB\\u0EAD-\\u0EB0\\u0EB2\\u0EB3\\u0EBD\\u0EC0-\\u0EC4\\u0EC6\\u0EDC-\\u0EDF\\u0F00\\u0F40-\\u0F47\\u0F49-\\u0F6C\\u0F88-\\u0F8C\\u1000-\\u102A\\u103F\\u1050-\\u1055\\u105A-\\u105D\\u1061\\u1065\\u1066\\u106E-\\u1070\\u1075-\\u1081\\u108E\\u10A0-\\u10C5\\u10C7\\u10CD\\u10D0-\\u10FA\\u10FC-\\u1248\\u124A-\\u124D\\u1250-\\u1256\\u1258\\u125A-\\u125D\\u1260-\\u1288\\u128A-\\u128D\\u1290-\\u12B0\\u12B2-\\u12B5\\u12B8-\\u12BE\\u12C0\\u12C2-\\u12C5\\u12C8-\\u12D6\\u12D8-\\u1310\\u1312-\\u1315\\u1318-\\u135A\\u1380-\\u138F\\u13A0-\\u13F4\\u1401-\\u166C\\u166F-\\u167F\\u1681-\\u169A\\u16A0-\\u16EA\\u16EE-\\u16F0\\u1700-\\u170C\\u170E-\\u1711\\u1720-\\u1731\\u1740-\\u1751\\u1760-\\u176C\\u176E-\\u1770\\u1780-\\u17B3\\u17D7\\u17DC\\u1820-\\u1877\\u1880-\\u18A8\\u18AA\\u18B0-\\u18F5\\u1900-\\u191C\\u1950-\\u196D\\u1970-\\u1974\\u1980-\\u19AB\\u19C1-\\u19C7\\u1A00-\\u1A16\\u1A20-\\u1A54\\u1AA7\\u1B05-\\u1B33\\u1B45-\\u1B4B\\u1B83-\\u1BA0\\u1BAE\\u1BAF\\u1BBA-\\u1BE5\\u1C00-\\u1C23\\u1C4D-\\u1C4F\\u1C5A-\\u1C7D\\u1CE9-\\u1CEC\\u1CEE-\\u1CF1\\u1CF5\\u1CF6\\u1D00-\\u1DBF\\u1E00-\\u1F15\\u1F18-\\u1F1D\\u1F20-\\u1F45\\u1F48-\\u1F4D\\u1F50-\\u1F57\\u1F59\\u1F5B\\u1F5D\\u1F5F-\\u1F7D\\u1F80-\\u1FB4\\u1FB6-\\u1FBC\\u1FBE\\u1FC2-\\u1FC4\\u1FC6-\\u1FCC\\u1FD0-\\u1FD3\\u1FD6-\\u1FDB\\u1FE0-\\u1FEC\\u1FF2-\\u1FF4\\u1FF6-\\u1FFC\\u2071\\u207F\\u2090-\\u209C\\u2102\\u2107\\u210A-\\u2113\\u2115\\u2119-\\u211D\\u2124\\u2126\\u2128\\u212A-\\u212D\\u212F-\\u2139\\u213C-\\u213F\\u2145-\\u2149\\u214E\\u2160-\\u2188\\u2C00-\\u2C2E\\u2C30-\\u2C5E\\u2C60-\\u2CE4\\u2CEB-\\u2CEE\\u2CF2\\u2CF3\\u2D00-\\u2D25\\u2D27\\u2D2D\\u2D30-\\u2D67\\u2D6F\\u2D80-\\u2D96\\u2DA0-\\u2DA6\\u2DA8-\\u2DAE\\u2DB0-\\u2DB6\\u2DB8-\\u2DBE\\u2DC0-\\u2DC6\\u2DC8-\\u2DCE\\u2DD0-\\u2DD6\\u2DD8-\\u2DDE\\u2E2F\\u3005-\\u3007\\u3021-\\u3029\\u3031-\\u3035\\u3038-\\u303C\\u3041-\\u3096\\u309D-\\u309F\\u30A1-\\u30FA\\u30FC-\\u30FF\\u3105-\\u312D\\u3131-\\u318E\\u31A0-\\u31BA\\u31F0-\\u31FF\\u3400-\\u4DB5\\u4E00-\\u9FCC\\uA000-\\uA48C\\uA4D0-\\uA4FD\\uA500-\\uA60C\\uA610-\\uA61F\\uA62A\\uA62B\\uA640-\\uA66E\\uA67F-\\uA697\\uA6A0-\\uA6EF\\uA717-\\uA71F\\uA722-\\uA788\\uA78B-\\uA78E\\uA790-\\uA793\\uA7A0-\\uA7AA\\uA7F8-\\uA801\\uA803-\\uA805\\uA807-\\uA80A\\uA80C-\\uA822\\uA840-\\uA873\\uA882-\\uA8B3\\uA8F2-\\uA8F7\\uA8FB\\uA90A-\\uA925\\uA930-\\uA946\\uA960-\\uA97C\\uA984-\\uA9B2\\uA9CF\\uAA00-\\uAA28\\uAA40-\\uAA42\\uAA44-\\uAA4B\\uAA60-\\uAA76\\uAA7A\\uAA80-\\uAAAF\\uAAB1\\uAAB5\\uAAB6\\uAAB9-\\uAABD\\uAAC0\\uAAC2\\uAADB-\\uAADD\\uAAE0-\\uAAEA\\uAAF2-\\uAAF4\\uAB01-\\uAB06\\uAB09-\\uAB0E\\uAB11-\\uAB16\\uAB20-\\uAB26\\uAB28-\\uAB2E\\uABC0-\\uABE2\\uAC00-\\uD7A3\\uD7B0-\\uD7C6\\uD7CB-\\uD7FB\\uF900-\\uFA6D\\uFA70-\\uFAD9\\uFB00-\\uFB06\\uFB13-\\uFB17\\uFB1D\\uFB1F-\\uFB28\\uFB2A-\\uFB36\\uFB38-\\uFB3C\\uFB3E\\uFB40\\uFB41\\uFB43\\uFB44\\uFB46-\\uFBB1\\uFBD3-\\uFD3D\\uFD50-\\uFD8F\\uFD92-\\uFDC7\\uFDF0-\\uFDFB\\uFE70-\\uFE74\\uFE76-\\uFEFC\\uFF21-\\uFF3A\\uFF41-\\uFF5A\\uFF66-\\uFFBE\\uFFC2-\\uFFC7\\uFFCA-\\uFFCF\\uFFD2-\\uFFD7\\uFFDA-\\uFFDC';
export default { unicodeLetters };
/**
* A regex that matches all single quotes in a string
*/
export const allSingleQuotes = /'/g;
export default { unicodeLetters, allSingleQuotes };

15
app/assets/javascripts/lib/utils/text_utility.js
View File

@ -4,6 +4,7 @@ import {
TRUNCATE_WIDTH_DEFAULT_WIDTH,
TRUNCATE_WIDTH_DEFAULT_FONT_SIZE,
} from '~/lib/utils/constants';
import { allSingleQuotes } from '~/lib/utils/regexp';
/**
* Adds a , to a string composed by numbers, at every 3 chars.
@ -479,3 +480,17 @@ export const markdownConfig = {
ALLOWED_ATTR: ['class', 'style', 'href', 'src'],
ALLOW_DATA_ATTR: false,
};
/**
* Escapes a string into a shell string, for example
* when you want to give a user the command to checkout
* a branch.
*
* It replaces all single-quotes with an escaped "'\''"
* that is interpreted by shell as a single-quote. It also
* encapsulates the string in single-quotes.
*
* If the branch is `fix-'bug-behavior'`, that should be
* escaped to `'fix-'\''bug-behavior'\'''`.
*/
export const escapeShellString = (str) => `'${str.replace(allSingleQuotes, () => "'\\''")}'`;

4
app/assets/javascripts/pages/profiles/two_factor_auths/index.js
View File

@ -1,5 +1,5 @@
import { mount2faRegistration } from '~/authentication/mount_2fa';
import { initRecoveryCodes } from '~/authentication/two_factor_auth';
import { initRecoveryCodes, initManageTwoFactorForm } from '~/authentication/two_factor_auth';
import { parseBoolean } from '~/lib/utils/common_utils';
const twoFactorNode = document.querySelector('.js-two-factor-auth');
@ -14,3 +14,5 @@ if (skippable) {
mount2faRegistration();
initRecoveryCodes();
initManageTwoFactorForm();

1
app/assets/javascripts/persistent_user_callouts.js
View File

@ -9,6 +9,7 @@ const PERSISTENT_USER_CALLOUTS = [
'.js-registration-enabled-callout',
'.js-new-user-signups-cap-reached',
'.js-eoa-bronze-plan-banner',
'.js-security-newsletter-callout',
];
const initCallouts = () => {

2
app/assets/javascripts/users_select/index.js
View File

@ -842,7 +842,7 @@ UsersSelect.prototype.renderApprovalRules = function (elsClassName, approvalRule
const [rule] = approvalRules;
const countText = sprintf(__('(+%{count}&nbsp;rules)'), { count });
const renderApprovalRulesCount = count > 1 ? `<span class="ml-1">${countText}</span>` : '';
const ruleName = rule.rule_type === 'code_owner' ? __('Code Owner') : rule.name;
const ruleName = rule.rule_type === 'code_owner' ? __('Code Owner') : escape(rule.name);
return `<div class="gl-display-flex gl-font-sm">
<span class="gl-text-truncate" title="${ruleName}">${ruleName}</span>

22
app/assets/javascripts/vue_merge_request_widget/components/mr_widget_how_to_merge_modal.vue
View File

@ -1,6 +1,7 @@
<script>
/* eslint-disable @gitlab/require-i18n-strings */
import { GlModal, GlLink, GlSprintf } from '@gitlab/ui';
import { escapeShellString } from '~/lib/utils/text_utility';
import { __ } from '~/locale';
import ClipboardButton from '~/vue_shared/components/clipboard_button.vue';
@ -75,20 +76,31 @@ export default {
},
computed: {
mergeInfo1() {
const escapedOriginBranch = escapeShellString(`origin/${this.sourceBranch}`);
return this.isFork
? `git fetch "${this.sourceProjectDefaultUrl}" ${this.sourceBranch}\ngit checkout -b "${this.sourceProjectPath}-${this.sourceBranch}" FETCH_HEAD`
: `git fetch origin\ngit checkout -b "${this.sourceBranch}" "origin/${this.sourceBranch}"`;
? `git fetch "${this.sourceProjectDefaultUrl}" ${this.escapedSourceBranch}\ngit checkout -b ${this.escapedForkBranch} FETCH_HEAD`
: `git fetch origin\ngit checkout -b ${this.escapedSourceBranch} ${escapedOriginBranch}`;
},
mergeInfo2() {
return this.isFork
? `git fetch origin\ngit checkout "${this.targetBranch}"\ngit merge --no-ff "${this.sourceProjectPath}-${this.sourceBranch}"`
: `git fetch origin\ngit checkout "${this.targetBranch}"\ngit merge --no-ff "${this.sourceBranch}"`;
? `git fetch origin\ngit checkout ${this.escapedTargetBranch}\ngit merge --no-ff ${this.escapedForkBranch}`
: `git fetch origin\ngit checkout ${this.escapedTargetBranch}\ngit merge --no-ff ${this.escapedSourceBranch}`;
},
mergeInfo3() {
return this.canMerge
? `git push origin "${this.targetBranch}"`
? `git push origin ${this.escapedTargetBranch}`
: __('Note that pushing to GitLab requires write access to this repository.');
},
escapedForkBranch() {
return escapeShellString(`${this.sourceProjectPath}-${this.sourceBranch}`);
},
escapedTargetBranch() {
return escapeShellString(this.targetBranch);
},
escapedSourceBranch() {
return escapeShellString(this.sourceBranch);
},
},
};
</script>

8
app/controllers/admin/dashboard_controller.rb
View File

@ -15,7 +15,13 @@ class Admin::DashboardController < Admin::ApplicationController
@groups = Group.order_id_desc.with_route.limit(10)
@notices = Gitlab::ConfigChecker::PumaRuggedChecker.check
@notices += Gitlab::ConfigChecker::ExternalDatabaseChecker.check
@redis_versions = [Gitlab::Redis::Queues, Gitlab::Redis::SharedState, Gitlab::Redis::Cache, Gitlab::Redis::TraceChunks].map(&:version).uniq
@redis_versions = [
Gitlab::Redis::Queues,
Gitlab::Redis::SharedState,
Gitlab::Redis::Cache,
Gitlab::Redis::TraceChunks,
Gitlab::Redis::RateLimiting
].map(&:version).uniq
end
# rubocop: enable CodeReuse/ActiveRecord

7
app/controllers/admin/users_controller.rb
View File

@ -45,10 +45,11 @@ class Admin::UsersController < Admin::ApplicationController
end
def impersonate
if can?(user, :log_in)
if can?(user, :log_in) && !impersonation_in_progress?
session[:impersonator_id] = current_user.id
warden.set_user(user, scope: :user)
clear_access_token_session_keys!
log_impersonation_event
@ -57,7 +58,9 @@ class Admin::UsersController < Admin::ApplicationController
redirect_to root_path
else
flash[:alert] =
if user.blocked?
if impersonation_in_progress?
_("You are already impersonating another user")
elsif user.blocked?
_("You cannot impersonate a blocked user")
elsif user.internal?
_("You cannot impersonate an internal user")

19
app/controllers/concerns/impersonation.rb
View File

@ -3,6 +3,12 @@
module Impersonation
include Gitlab::Utils::StrongMemoize
SESSION_KEYS_TO_DELETE = %w(
github_access_token gitea_access_token gitlab_access_token
bitbucket_token bitbucket_refresh_token bitbucket_server_personal_access_token
bulk_import_gitlab_access_token fogbugz_token
).freeze
def current_user
user = super
@ -14,7 +20,7 @@ module Impersonation
protected
def check_impersonation_availability
return unless session[:impersonator_id]
return unless impersonation_in_progress?
unless Gitlab.config.gitlab.impersonation_enabled
stop_impersonation
@ -27,14 +33,25 @@ module Impersonation
warden.set_user(impersonator, scope: :user)
session[:impersonator_id] = nil
clear_access_token_session_keys!
current_user
end
def impersonation_in_progress?
session[:impersonator_id].present?
end
def log_impersonation_event
Gitlab::AppLogger.info("User #{impersonator.username} has stopped impersonating #{current_user.username}")
end
def clear_access_token_session_keys!
access_tokens_keys = session.keys & SESSION_KEYS_TO_DELETE
access_tokens_keys.each { |key| session.delete(key) }
end
def impersonator
strong_memoize(:impersonator) do
User.find(session[:impersonator_id]) if session[:impersonator_id]

1
app/controllers/health_controller.rb
View File

@ -16,6 +16,7 @@ class HealthController < ActionController::Base
Gitlab::HealthChecks::Redis::QueuesCheck,
Gitlab::HealthChecks::Redis::SharedStateCheck,
Gitlab::HealthChecks::Redis::TraceChunksCheck,
Gitlab::HealthChecks::Redis::RateLimitingCheck,
Gitlab::HealthChecks::GitalyCheck
].freeze

6
app/controllers/import/gitea_controller.rb
View File

@ -66,11 +66,13 @@ class Import::GiteaController < Import::GithubController
override :client_options
def client_options
{ host: provider_url, api_version: 'v1' }
verified_url, provider_hostname = verify_blocked_uri
{ host: verified_url.scheme == 'https' ? provider_url : verified_url.to_s, api_version: 'v1', hostname: provider_hostname }
end
def verify_blocked_uri
Gitlab::UrlBlocker.validate!(
@verified_url_and_hostname ||= Gitlab::UrlBlocker.validate!(
provider_url,
allow_localhost: allow_local_requests?,
allow_local_network: allow_local_requests?,

8
app/controllers/profiles/passwords_controller.rb
View File

@ -47,6 +47,8 @@ class Profiles::PasswordsController < Profiles::ApplicationController
password_attributes[:password_automatically_set] = false
unless @user.password_automatically_set || @user.valid_password?(user_params[:current_password])
handle_invalid_current_password_attempt!
redirect_to edit_profile_password_path, alert: _('You must provide a valid current password')
return
end
@ -85,6 +87,12 @@ class Profiles::PasswordsController < Profiles::ApplicationController
render_404 unless @user.allow_password_authentication?
end
def handle_invalid_current_password_attempt!
Gitlab::AppLogger.info(message: 'Invalid current password when attempting to update user password', username: @user.username, ip: request.remote_ip)
@user.increment_failed_attempts!
end
def user_params
params.require(:user).permit(:current_password, :password, :password_confirmation)
end

10
app/controllers/profiles/two_factor_auths_controller.rb
View File

@ -3,6 +3,8 @@
class Profiles::TwoFactorAuthsController < Profiles::ApplicationController
skip_before_action :check_two_factor_requirement
before_action :ensure_verified_primary_email, only: [:show, :create]
before_action :validate_current_password, only: [:create, :codes, :destroy]
before_action do
push_frontend_feature_flag(:webauthn)
end
@ -134,6 +136,14 @@ class Profiles::TwoFactorAuthsController < Profiles::ApplicationController
private
def validate_current_password
return if current_user.valid_password?(params[:current_password])
current_user.increment_failed_attempts!
redirect_to profile_two_factor_auth_path, alert: _('You must provide a valid current password')
end
def build_qr_code
uri = current_user.otp_provisioning_uri(account_string, issuer: issuer_host)
RQRCode.render_qrcode(uri, :svg, level: :m, unit: 3)

4
app/controllers/projects/project_members_controller.rb
View File

@ -34,13 +34,13 @@ class Projects::ProjectMembersController < Projects::ApplicationController
end
def import
@projects = current_user.authorized_projects.order_id_desc
@projects = Project.visible_to_user_and_access_level(current_user, Gitlab::Access::MAINTAINER).order_id_desc
end
def apply_import
source_project = Project.find(params[:source_project_id])
if can?(current_user, :read_project_member, source_project)
if can?(current_user, :admin_project_member, source_project)
status = @project.team.import(source_project, current_user)
notice = status ? "Successfully imported" : "Import failed"
else

5
app/controllers/projects_controller.rb
View File

@ -19,6 +19,7 @@ class ProjectsController < Projects::ApplicationController
before_action :redirect_git_extension, only: [:show]
before_action :project, except: [:index, :new, :create, :resolve]
before_action :repository, except: [:index, :new, :create, :resolve]
before_action :verify_git_import_enabled, only: [:create]
before_action :project_export_enabled, only: [:export, :download_export, :remove_export, :generate_new_export]
before_action :present_project, only: [:edit]
before_action :authorize_download_code!, only: [:refs]
@ -496,6 +497,10 @@ class ProjectsController < Projects::ApplicationController
url_for(safe_params)
end
def verify_git_import_enabled
render_404 if project_params[:import_url] && !git_import_enabled?
end
def project_export_enabled
render_404 unless Gitlab::CurrentSettings.project_export_enabled?
end

6
app/controllers/uploads_controller.rb
View File

@ -36,14 +36,10 @@ class UploadsController < ApplicationController
end
def find_model
return unless params[:id]
upload_model_class.find(params[:id])
end
def authorize_access!
return unless model
authorized =
case model
when Note
@ -68,8 +64,6 @@ class UploadsController < ApplicationController
end
def authorize_create_access!
return unless model
authorized =
case model
when User

2
app/graphql/types/group_invitation_type.rb
View File

@ -3,7 +3,7 @@
module Types
class GroupInvitationType < BaseObject
expose_permissions Types::PermissionTypes::Group
authorize :read_group
authorize :admin_group
implements InvitationInterface

2
app/graphql/types/project_invitation_type.rb
View File

@ -9,7 +9,7 @@ module Types
implements InvitationInterface
authorize :read_project
authorize :admin_project
field :project, Types::ProjectType, null: true,
description: 'Project ID for the project of the invitation.'

5
app/helpers/external_link_helper.rb
View File

@ -1,9 +1,12 @@
# frozen_string_literal: true
module ExternalLinkHelper
include ActionView::Helpers::TextHelper
def external_link(body, url, options = {})
link_to url, { target: '_blank', rel: 'noopener noreferrer' }.merge(options) do
link = link_to url, { target: '_blank', rel: 'noopener noreferrer' }.merge(options) do
"#{body}#{sprite_icon('external-link', css_class: 'gl-ml-1')}".html_safe
end
sanitize(link, tags: %w(a svg use), attributes: %w(target rel data-testid class href).concat(options.stringify_keys.keys))
end
end

2
app/helpers/icons_helper.rb
View File

@ -44,7 +44,7 @@ module IconsHelper
content_tag(
:svg,
content_tag(:use, '', { 'xlink:href' => "#{sprite_icon_path}##{icon_name}" } ),
content_tag(:use, '', { 'href' => "#{sprite_icon_path}##{icon_name}" } ),
class: css_classes.empty? ? nil : css_classes.join(' '),
data: { testid: "#{icon_name}-icon" }
)

6
app/helpers/user_callouts_helper.rb
View File

@ -10,6 +10,7 @@ module UserCalloutsHelper
REGISTRATION_ENABLED_CALLOUT = 'registration_enabled_callout'
UNFINISHED_TAG_CLEANUP_CALLOUT = 'unfinished_tag_cleanup_callout'
INVITE_MEMBERS_BANNER = 'invite_members_banner'
SECURITY_NEWSLETTER_CALLOUT = 'security_newsletter_callout'
def show_gke_cluster_integration_callout?(project)
active_nav_link?(controller: sidebar_operations_paths) &&
@ -64,6 +65,11 @@ module UserCalloutsHelper
!multiple_members?(group)
end
def show_security_newsletter_user_callout?
current_user&.admin? &&
!user_dismissed?(SECURITY_NEWSLETTER_CALLOUT)
end
private
def user_dismissed?(feature_name, ignore_dismissal_earlier_than = nil)

1
app/models/project_statistics.rb
View File

@ -31,7 +31,6 @@ class ProjectStatistics < ApplicationRecord
scope :for_project_ids, ->(project_ids) { where(project_id: project_ids) }
scope :for_namespaces, -> (namespaces) { where(namespace: namespaces) }
scope :with_any_ci_minutes_used, -> { where.not(shared_runners_seconds: 0) }
def total_repository_size
repository_size + lfs_objects_size

3
app/models/user.rb
View File

@ -1886,7 +1886,8 @@ class User < ApplicationRecord
def password_expired_if_applicable?
return false if bot?
return false unless password_expired? && password_automatically_set?
return false unless password_expired?
return false if password_automatically_set?
return false unless allow_password_authentication?
true

3
app/models/user_callout.rb
View File

@ -36,7 +36,8 @@ class UserCallout < ApplicationRecord
trial_status_reminder_d3: 35, # EE-only
security_configuration_devops_alert: 36, # EE-only
profile_personal_access_token_expiry: 37, # EE-only
terraform_notification_dismissed: 38
terraform_notification_dismissed: 38,
security_newsletter_callout: 39
}
validates :feature_name,

11
app/services/projects/destroy_service.rb
View File

@ -117,6 +117,7 @@ module Projects
trash_relation_repositories!
trash_project_repositories!
destroy_web_hooks!
destroy_project_bots!
# Rails attempts to load all related records into memory before
# destroying: https://github.com/rails/rails/issues/22510
@ -149,6 +150,16 @@ module Projects
end
end
# The project can have multiple project bots with personal access tokens generated.
# We need to remove them when a project is deleted
# rubocop: disable CodeReuse/ActiveRecord
def destroy_project_bots!
project.members.includes(:user).references(:user).merge(User.project_bot).each do |member|
Users::DestroyService.new(current_user).execute(member.user, skip_authorization: true)
end
end
# rubocop: enable CodeReuse/ActiveRecord
def remove_registry_tags
return true unless Gitlab.config.registry.enabled
return false unless remove_legacy_registry_tags

14
app/views/admin/dashboard/_security_newsletter_callout.html.haml Normal file
View File

@ -0,0 +1,14 @@
- return unless show_security_newsletter_user_callout?
= render 'shared/global_alert',
title: s_('AdminArea|Get security updates from GitLab and stay up to date'),
variant: :tip,
alert_class: 'js-security-newsletter-callout',
is_contained: true,
alert_data: { feature_id: UserCalloutsHelper::SECURITY_NEWSLETTER_CALLOUT, dismiss_endpoint: user_callouts_path, defer_links: 'true' },
close_button_data: { testid: 'close-security-newsletter-callout' } do
.gl-alert-body
= s_('AdminArea|Sign up for the GitLab Security Newsletter to get notified for security updates.')
.gl-alert-actions
= link_to 'https://about.gitlab.com/company/preference-center/', target: '_blank', rel: 'noreferrer noopener', class: 'deferred-link gl-alert-action btn-confirm btn-md gl-button' do
= s_('AdminArea|Sign up for the GitLab newsletter')

1
app/views/admin/dashboard/index.html.haml
View File

@ -4,6 +4,7 @@
- billable_users_link_start = '<a href="%{url}" target="_blank" rel="noopener noreferrer nofollow">'.html_safe % { url: billable_users_url }
= render_if_exists 'shared/qrtly_reconciliation_alert'
= render 'admin/dashboard/security_newsletter_callout'
- if @notices
- @notices.each do |notice|

13
app/views/profiles/two_factor_auths/show.html.haml
View File

@ -17,13 +17,7 @@
= _("You've already enabled two-factor authentication using one time password authenticators. In order to register a different device, you must first disable two-factor authentication.")
%p
= _('If you lose your recovery codes you can generate new ones, invalidating all previous codes.')
%div
= link_to _('Disable two-factor authentication'), profile_two_factor_auth_path,
method: :delete,
data: { confirm: webauthn_enabled ? _('Are you sure? This will invalidate your registered applications and U2F / WebAuthn devices.') : _('Are you sure? This will invalidate your registered applications and U2F devices.') },
class: 'gl-button btn btn-danger gl-mr-3'
= form_tag codes_profile_two_factor_auth_path, {style: 'display: inline-block', method: :post} do |f|
= submit_tag _('Regenerate recovery codes'), class: 'gl-button btn btn-default'
.js-manage-two-factor-form{ data: { webauthn_enabled: webauthn_enabled, profile_two_factor_auth_path: profile_two_factor_auth_path, profile_two_factor_auth_method: 'delete', codes_profile_two_factor_auth_path: codes_profile_two_factor_auth_path, codes_profile_two_factor_auth_method: 'post' } }
- else
%p
@ -53,6 +47,11 @@
.form-group
= label_tag :pin_code, _('Pin code'), class: "label-bold"
= text_field_tag :pin_code, nil, class: "form-control gl-form-input", required: true, data: { qa_selector: 'pin_code_field' }
.form-group
= label_tag :current_password, _('Current password'), class: 'label-bold'
= password_field_tag :current_password, nil, required: true, class: 'form-control gl-form-input', data: { qa_selector: 'current_password_field' }
%p.form-text.text-muted
= _('Your current password is required to register a two-factor authenticator app.')
.gl-mt-3
= submit_tag _('Register with two-factor app'), class: 'gl-button btn btn-confirm', data: { qa_selector: 'register_2fa_app_button' }

1
app/views/projects/_export.html.haml
View File

@ -16,6 +16,7 @@
%li= _('Job logs and artifacts')
%li= _('Container registry images')
%li= _('CI variables')
%li= _('Pipeline triggers')
%li= _('Webhooks')
%li= _('Any encrypted tokens')
- if project.export_status == :finished

158
config/README.md
View File

@ -1,13 +1,13 @@
# Configuration files Documentation
Note that most configuration files (`config/*.*`) committed into
[gitlab-ce](https://gitlab.com/gitlab-org/gitlab-foss) **will not be used** for
[gitlab-foss](https://gitlab.com/gitlab-org/gitlab-foss) **will not be used** for
[omnibus-gitlab](https://gitlab.com/gitlab-org/omnibus-gitlab). Configuration
files committed into gitlab-ce are only used for development.
files committed into gitlab-foss are only used for development.
## gitlab.yml
You can find most of GitLab configuration settings here.
You can find most of the GitLab configuration settings here.
## mail_room.yml
@ -21,7 +21,7 @@ This file is called `resque.yml` for historical reasons. We are **NOT**
using Resque at the moment. It is used to specify Redis configuration
values when a single database instance of Redis is desired.
# Advanced Redis configuration files
## Advanced Redis configuration files
In more advanced configurations of Redis key-value storage, it is desirable
to separate the keys by lifecycle and intended use to ease provisioning and
@ -40,7 +40,7 @@ If desired, the routing URL provided by these settings can be used with:
2. TCP port number for each Redis instance desired
3. `database number` for each Redis instance desired
## Example URL attribute formats for GitLab Redis `.yml` configuration files
### Example URL attribute formats for GitLab Redis `.yml` configuration files
* Unix Socket, default Redis database (0)
* `url: unix:/path/to/redis.sock`
* `url: unix:/path/to/redis.sock?db=`
@ -52,129 +52,37 @@ If desired, the routing URL provided by these settings can be used with:
* TCP Socket for Redis on remote host `myserver`, port 6379, database 33
* `url: redis://:mynewpassword@myserver:6379/33`
## redis.cache.yml
## Available configuration files
If configured, `redis.cache.yml` overrides the
`resque.yml` settings to configure the Redis database instance
used for `Rails.cache` and other volatile non-persistent data which enhances
the performance of GitLab.
Settings here can be overridden by the environment variable
`GITLAB_REDIS_CACHE_CONFIG_FILE` which provides
an alternate location for configuration settings.
The Redis instances that can be configured are described in the table below. The
order of precedence for configuration is described below, where `$NAME` and
`$FALLBACK_NAME` are the upper-cased instance names from the table, and `$name`
and `$fallback_name` are the lower-cased versions:
The order of precedence for the URL used to connect to the Redis instance
used for `cache` is:
1. URL from a configuration file pointed to by the
`GITLAB_REDIS_CACHE_CONFIG_FILE` environment variable
2. URL from `redis.cache.yml`
3. URL from a configuration file pointed to by the
`GITLAB_REDIS_CONFIG_FILE` environment variable
4. URL from `resque.yml`
5. `redis://localhost:6380`
1. The configuration file pointed to by the `GITLAB_REDIS_$NAME_CONFIG_FILE`
environment variable.
1. The configuration file `redis.$name.yml`.
1. **If a fallback instance is available**, the configuration file
`redis.$fallback_name.yml`.
1. The configuration file pointed to by the `GITLAB_REDIS_CONFIG_FILE`
environment variable.
1. The configuration file `resque.yml`.
The order of precedence for all other configuration settings for `cache`
are selected from only the first of the following files found (if a setting
is not provided in an earlier file, the remainder of the files are not
searched):
1. the configuration file pointed to by the
`GITLAB_REDIS_CACHE_CONFIG_FILE` environment variable
2. the configuration file `redis.cache.yml`
3. the configuration file pointed to by the
`GITLAB_REDIS_CONFIG_FILE` environment variable
4. the configuration file `resque.yml`
An example configuration file for Redis is in this directory under the name
`resque.yml.example`.
## redis.queues.yml
| Name | Fallback instance | Purpose |
| --- | --- | --- |
| `cache` | | Volatile non-persistent data |
| `queues` | | Background job processing queues |
| `shared_state` | | Persistent application state |
| `trace_chunks` | `shared_state` | [CI trace chunks](https://docs.gitlab.com/ee/administration/job_logs.html#incremental-logging-architecture) |
| `rate_limiting` | `cache` | [Rate limiting](https://docs.gitlab.com/ee/user/admin_area/settings/user_and_ip_rate_limits.html) state |
If configured, `redis.queues.yml` overrides the
`resque.yml` settings to configure the Redis database instance
used for clients of `::Gitlab::Redis::Queues`.
These queues are intended to be the foundation
of reliable inter-process communication between modules, whether on the same
host node, or within a cluster. The primary clients of the queues are
SideKiq, Mailroom, CI Runner, Workhorse, and push services. Settings here can
be overridden by the environment variable
`GITLAB_REDIS_QUEUES_CONFIG_FILE` which provides an alternate location for
configuration settings.
If no configuration is found, or no URL is found in the configuration
file, the default URL used is:
The order of precedence for the URL used to connect to the Redis instance
used for `queues` is:
1. URL from a configuration file pointed to by the
`GITLAB_REDIS_QUEUES_CONFIG_FILE` environment variable
2. URL from `redis.queues.yml`
3. URL from a configuration file pointed to by the
`GITLAB_REDIS_CONFIG_FILE` environment variable
4. URL from `resque.yml`
5. `redis://localhost:6381`
The order of precedence for all other configuration settings for `queues`
are selected from only the first of the following files found (if a setting
is not provided in an earlier file, the remainder of the files are not
searched):
1. the configuration file pointed to by the
`GITLAB_REDIS_QUEUES_CONFIG_FILE` environment variable
2. the configuration file `redis.queues.yml`
3. the configuration file pointed to by the
`GITLAB_REDIS_CONFIG_FILE` environment variable
4. the configuration file `resque.yml`
## redis.shared_state.yml
If configured, `redis.shared_state.yml` overrides the
`resque.yml` settings to configure the Redis database instance
used for clients of `::Gitlab::Redis::SharedState` such as session state,
and rate limiting.
Settings here can be overridden by the environment variable
`GITLAB_REDIS_SHARED_STATE_CONFIG_FILE` which provides
an alternate location for configuration settings.
The order of precedence for the URL used to connect to the Redis instance
used for `shared_state` is:
1. URL from a configuration file pointed to by the
`GITLAB_REDIS_SHARED_STATE_CONFIG_FILE` environment variable
2. URL from `redis.shared_state.yml`
3. URL from a configuration file pointed to by the
`GITLAB_REDIS_CONFIG_FILE` environment variable
4. URL from `resque.yml`
5. `redis://localhost:6382`
The order of precedence for all other configuration settings for `shared_state`
are selected from only the first of the following files found (if a setting
is not provided in an earlier file, the remainder of the files are not
searched):
1. the configuration file pointed to by the
`GITLAB_REDIS_SHARED_STATE_CONFIG_FILE` environment variable
2. the configuration file `redis.shared_state.yml`
3. the configuration file pointed to by the
`GITLAB_REDIS_CONFIG_FILE` environment variable
4. the configuration file `resque.yml`
## redis.trace_chunks.yml
If configured, `redis.trace_chunks.yml` overrides the
`resque.yml` settings to configure the Redis database instance
used for clients of `::Gitlab::Redis::TraceChunks` which stores CI trace chunks.
Settings here can be overridden by the environment variable
`GITLAB_REDIS_TRACE_CHUNKS_CONFIG_FILE` which provides
an alternate location for configuration settings.
The order of precedence for the URL used to connect to the Redis instance
used for `trace_chunks` is:
1. URL from a configuration file pointed to by the
`GITLAB_REDIS_TRACE_CHUNKS_CONFIG_FILE` environment variable
2. URL from `redis.trace_chunks.yml`
3. URL from a configuration file pointed to by the
`GITLAB_REDIS_CONFIG_FILE` environment variable
4. URL from `resque.yml`
5. `redis://localhost:6383`
The order of precedence for all other configuration settings for `trace_chunks`
are selected from only the first of the following files found (if a setting
is not provided in an earlier file, the remainder of the files are not
searched):
1. the configuration file pointed to by the
`GITLAB_REDIS_TRACE_CHUNKS_CONFIG_FILE` environment variable
2. the configuration file `redis.trace_chunks.yml`
3. the configuration file pointed to by the
`GITLAB_REDIS_CONFIG_FILE` environment variable
4. the configuration file `resque.yml`
1. `redis://localhost:6380` for `cache`.
1. `redis://localhost:6381` for `queues`.
1. `redis://localhost:6382` for `shared_state`.
1. The URL from the fallback instance for all other instances.

12
config/application.rb
View File

@ -23,6 +23,8 @@ module Gitlab
require_dependency Rails.root.join('lib/gitlab/redis/cache')
require_dependency Rails.root.join('lib/gitlab/redis/queues')
require_dependency Rails.root.join('lib/gitlab/redis/shared_state')
require_dependency Rails.root.join('lib/gitlab/redis/trace_chunks')
require_dependency Rails.root.join('lib/gitlab/redis/rate_limiting')
require_dependency Rails.root.join('lib/gitlab/current_settings')
require_dependency Rails.root.join('lib/gitlab/middleware/read_only')
require_dependency Rails.root.join('lib/gitlab/middleware/basic_health_check')
@ -370,15 +372,7 @@ module Gitlab
end
# Use caching across all environments
# Full list of options:
# https://api.rubyonrails.org/classes/ActiveSupport/Cache/RedisCacheStore.html#method-c-new
caching_config_hash = {}
caching_config_hash[:redis] = Gitlab::Redis::Cache.pool
caching_config_hash[:compress] = Gitlab::Utils.to_boolean(ENV.fetch('ENABLE_REDIS_CACHE_COMPRESSION', '1'))
caching_config_hash[:namespace] = Gitlab::Redis::Cache::CACHE_NAMESPACE
caching_config_hash[:expires_in] = 2.weeks # Cache should not grow forever
config.cache_store = :redis_cache_store, caching_config_hash
config.cache_store = :redis_cache_store, Gitlab::Redis::Cache.active_support_config
config.active_job.queue_adapter = :sidekiq

1
config/initializers/7_redis.rb
View File

@ -11,3 +11,4 @@ Gitlab::Redis::Cache.with { nil }
Gitlab::Redis::Queues.with { nil }
Gitlab::Redis::SharedState.with { nil }
Gitlab::Redis::TraceChunks.with { nil }
Gitlab::Redis::RateLimiting.with { nil }

5
config/initializers/doorkeeper.rb
View File

@ -51,6 +51,11 @@ Doorkeeper.configure do
# Issue access tokens with refresh token (disabled by default)
use_refresh_token
# Forbids creating/updating applications with arbitrary scopes that are
# not in configuration, i.e. `default_scopes` or `optional_scopes`.
# (disabled by default)
enforce_configured_scopes
# Forces the usage of the HTTPS protocol in non-native redirect uris (enabled
# by default in non-development environments). OAuth2 delegates security in
# communication to the HTTPS protocol so it is wise to keep this enabled.

42
config/initializers_before_autoloader/100_patch_omniauth_oauth2.rb
View File

@ -1,14 +1,46 @@
# frozen_string_literal: true
# See https://github.com/omniauth/omniauth-oauth2/blob/v1.7.1/lib/omniauth/strategies/oauth2.rb#L84-L101
# for the original version of this code.
#
# Note: We need to override `callback_phase` directly (instead of using a module with `include` or `prepend`),
# because the method has a `super` call which needs to go to the `OmniAuth::Strategy` module,
# and it also deletes `omniauth.state` from the session as a side effect.
module OmniAuth
module Strategies
class OAuth2
alias_method :original_callback_phase, :callback_phase
# Monkey patch until PR is merged and released upstream
# https://github.com/omniauth/omniauth-oauth2/pull/129
def callback_phase
original_callback_phase
error = request.params["error_reason"].presence || request.params["error"].presence
# Monkey patch #1:
#
# Swap the order of these conditions around so the `state` param is verified *first*,
# before using the error params returned by the provider.
#
# This avoids content spoofing attacks by crafting a URL with malicious messages,
# because the `state` param is only present in the session after a valid OAuth2 authentication flow.
if !options.provider_ignores_state && (request.params["state"].to_s.empty? || request.params["state"] != session.delete("omniauth.state"))
fail!(:csrf_detected, CallbackError.new(:csrf_detected, "CSRF detected"))
elsif error
fail!(error, CallbackError.new(request.params["error"], request.params["error_description"].presence || request.params["error_reason"].presence, request.params["error_uri"]))
else
self.access_token = build_access_token
self.access_token = access_token.refresh! if access_token.expired?
super
end
rescue ::OAuth2::Error, CallbackError => e
fail!(:invalid_credentials, e)
rescue ::Timeout::Error, ::Errno::ETIMEDOUT => e
fail!(:timeout, e)
rescue ::SocketError => e
fail!(:failed_to_connect, e)
# Monkey patch #2:
#
# Also catch errors from Faraday.
# See https://github.com/omniauth/omniauth-oauth2/pull/129
# and https://github.com/oauth-xx/oauth2/issues/152
#
# This can be removed with https://gitlab.com/gitlab-org/gitlab/-/issues/340933
rescue ::Faraday::TimeoutError, ::Faraday::ConnectionFailed => e
fail!(:timeout, e)
end

38
config/redis.cache.yml.example
View File

@ -1,38 +0,0 @@
# If you change this file in a merge request, please also create
# a merge request on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests
#
development:
url: redis://localhost:6379/10
#
# url: redis://localhost:6380
# sentinels:
# -
# host: localhost
# port: 26380 # point to sentinel, not to redis port
# -
# host: replica2
# port: 26380 # point to sentinel, not to redis port
test:
url: redis://localhost:6379/10
#
# url: redis://localhost:6380
production:
# Redis (single instance)
url: unix:/var/run/redis/redis.cache.sock
##
# Redis + Sentinel (for HA)
#
# Please read instructions carefully before using it as you may lose data:
# http://redis.io/topics/sentinel
#
# You must specify a list of a few sentinels that will handle client connection
# please read here for more information: https://docs.gitlab.com/ee/administration/redis/index.html
##
# url: redis://master:6380
# sentinels:
# -
# host: replica1
# port: 26380 # point to sentinel, not to redis port
# -
# host: replica2
# port: 26380 # point to sentinel, not to redis port

38
config/redis.queues.yml.example
View File

@ -1,38 +0,0 @@
# If you change this file in a merge request, please also create
# a merge request on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests
#
development:
url: redis://localhost:6379/11
#
# url: redis://localhost:6381
# sentinels:
# -
# host: localhost
# port: 26381 # point to sentinel, not to redis port
# -
# host: replica2
# port: 26381 # point to sentinel, not to redis port
test:
url: redis://localhost:6379/11
#
# url: redis://localhost:6381
production:
# Redis (single instance)
url: unix:/var/run/redis/redis.queues.sock
##
# Redis + Sentinel (for HA)
#
# Please read instructions carefully before using it as you may lose data:
# http://redis.io/topics/sentinel
#
# You must specify a list of a few sentinels that will handle client connection
# please read here for more information: https://docs.gitlab.com/ee/administration/redis/index.html
##
# url: redis://master:6381
# sentinels:
# -
# host: replica1
# port: 26381 # point to sentinel, not to redis port
# -
# host: replica2
# port: 26381 # point to sentinel, not to redis port

38
config/redis.shared_state.yml.example
View File

@ -1,38 +0,0 @@
# If you change this file in a merge request, please also create
# a merge request on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests
#
development:
url: redis://localhost:6379/12
#
# url: redis://localhost:6382
# sentinels:
# -
# host: localhost
# port: 26382 # point to sentinel, not to redis port
# -
# host: replica2
# port: 26382 # point to sentinel, not to redis port
test:
url: redis://localhost:6379/12
#
# url: redis://localhost:6382
production:
# Redis (single instance)
url: unix:/var/run/redis/redis.shared_state.sock
##
# Redis + Sentinel (for HA)
#
# Please read instructions carefully before using it as you may lose data:
# http://redis.io/topics/sentinel
#
# You must specify a list of a few sentinels that will handle client connection
# please read here for more information: https://docs.gitlab.com/ee/administration/redis/index.html
##
# url: redis://master:6382
# sentinels:
# -
# host: replica1
# port: 26382 # point to sentinel, not to redis port
# -
# host: replica2
# port: 26382 # point to sentinel, not to redis port

38
config/redis.trace_chunks.yml.example
View File

@ -1,38 +0,0 @@
# If you change this file in a merge request, please also create
# a merge request on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests
#
development:
url: redis://localhost:6379/13
#
# url: redis://localhost:6382
# sentinels:
# -
# host: localhost
# port: 26382 # point to sentinel, not to redis port
# -
# host: replica2
# port: 26382 # point to sentinel, not to redis port
test:
url: redis://localhost:6379/13
#
# url: redis://localhost:6382
production:
# Redis (single instance)
url: unix:/var/run/redis/redis.trace_chunks.sock
##
# Redis + Sentinel (for HA)
#
# Please read instructions carefully before using it as you may lose data:
# http://redis.io/topics/sentinel
#
# You must specify a list of a few sentinels that will handle client connection
# please read here for more information: https://docs.gitlab.com/ee/administration/redis/index.html
##
# url: redis://master:6382
# sentinels:
# -
# host: replica1
# port: 26382 # point to sentinel, not to redis port
# -
# host: replica2
# port: 26382 # point to sentinel, not to redis port

54
db/post_migrate/20210914095310_cleanup_orphan_project_access_tokens.rb Normal file
View File

@ -0,0 +1,54 @@
# frozen_string_literal: true
class CleanupOrphanProjectAccessTokens < Gitlab::Database::Migration[1.0]
disable_ddl_transaction!
TMP_INDEX_NAME = 'idx_users_on_user_type_project_bots_batched'
def up
users_table = define_batchable_model('users')
add_concurrent_index(:users, :id, name: TMP_INDEX_NAME, where: 'user_type = 6')
accumulated_orphans = []
users_table.where(user_type: 6).each_batch(of: 500) do |relation|
orphan_ids = relation.where("not exists(select 1 from members where members.user_id = users.id)").pluck(:id)
orphan_ids.each_slice(10) do |ids|
users_table.where(id: ids).update_all(state: 'deactivated')
end
accumulated_orphans += orphan_ids
end
schedule_deletion(accumulated_orphans)
ensure
remove_concurrent_index_by_name(:users, TMP_INDEX_NAME)
end
def down
remove_concurrent_index_by_name(:users, TMP_INDEX_NAME) if index_exists_by_name?(:users, TMP_INDEX_NAME)
end
private
def schedule_deletion(orphan_ids)
return unless deletion_worker
orphan_ids.each_slice(100) do |ids|
job_arguments = ids.map do |orphan_id|
[orphan_id, orphan_id, { skip_authorization: true }]
end
deletion_worker.bulk_perform_async(job_arguments)
end
rescue StandardError
# Ignore any errors or interface changes since this part of migration is optional
end
def deletion_worker
@deletion_worker = "DeleteUserWorker".safe_constantize unless defined?(@deletion_worker)
@deletion_worker
end
end

1
db/schema_migrations/20210914095310 Normal file
View File

@ -0,0 +1 @@
6fcf3ff9867df68f5e9603ae0311b29bec33aa5c5b826786b094ab0960ebcd90

36
doc/administration/configure.md
View File

@ -7,10 +7,44 @@ type: reference
# Configure your GitLab installation **(FREE SELF)**
Customize and configure your self-managed GitLab installation.
Customize and configure your self-managed GitLab installation. Here are some quick links to get you started:
- [Authentication](auth/index.md)
- [Configuration](../user/admin_area/index.md)
- [Repository storage](repository_storage_paths.md)
- [Geo](geo/index.md)
- [Packages](packages/index.md)
The following tables are intended to guide you to choose the right combination of capabilties based on your requirements. It is common to want the most
available, quickly recoverable, highly performant and fully data resilient solution. However, there are tradeoffs.
The tables lists features on the left and provides their capabilities to the right along with known trade-offs.
## Gitaly Capabilities
| | Availability | Recoverability | Data Resiliency | Performance | Risks/Trade-offs|
|-|--------------|----------------|-----------------|-------------|-----------------|
|Gitaly Cluster | Very high - tolerant of node failures | RTO for a single node of 10s with no manual intervention | Data is stored on multiple nodes | Good - While writes may take slightly longer due to voting, read distribution improves read speeds | **Trade-off** - Slight decrease in write speed for redundant, strongly-consistent storage solution. **Risks** - [Does not currently support snapshot backups](gitaly/index.md#snapshot-backup-and-recovery-limitations), GitLab backup task can be slow for large data sets |
|Gitaly Shards | Single storage location is a single point of failure | Would need to restore only shards which failed | Single point of failure | Good - can allocate repositories to shards to spread load | **Trade-off** - Need to manually configure repositories into different shards to balance loads / storage space **Risks** - Single point of failure relies on recovery process when single-node failure occurs |
|Gitaly + NFS | Single storage location is a single point of failure | Single node failure requires restoration from backup | Single point of failure | Average - NFS is not ideally suited to large quantities of small reads / writes which can have a detrimental impact on performance | **Trade-off** - Easy and familiar administration though NFS is not ideally suited to Git demands **Risks** - Many instances of NFS compatibility issues which provide very poor customer experiences |
## Geo Capabilities
If your availabity needs to span multiple zones or multiple locations, please read about [Geo](geo/index.md).
| | Availability | Recoverability | Data Resiliency | Performance | Risks/Trade-offs|
|-|--------------|----------------|-----------------|-------------|-----------------|
|Geo| Depends on the architecture of the Geo site. It is possible to deploy secondaries in single and multiple node configurations. | Eventually consistent. Recovery point depends on replication lag, which depends on a number of factors such as network speeds. Geo supports failover from a primary to secondary site using manual commands that are scriptable. | Geo currently replicates 100% of planned data types and verifies 50%. See [limitations table](geo/replication/datatypes.md#limitations-on-replicationverification) for more detail. | Improves read/clone times for users of a secondary. | Geo is not intended to replace other backup/restore solutions. Because of replication lag and the possibility of replicating bad data from a primary, we recommend that customers also take regular backups of their primary site and test the restore process. |
## Scenarios for failure modes and available mitigation paths
The following table outlines failure modes and mitigation paths for the product offerings detailed in the tables above. Note - Gitaly Cluster install assumes an odd number replication factor of 3 or greater
| Gitaly Mode | Loss of Single Gitaly Node | Application / Data Corruption | Regional Outage (Loss of Instance) | Notes |
| ----------- | -------------------------- | ----------------------------- | ---------------------------------- | ----- |
| Single Gitaly Node | Downtime - Must restore from backup | Downtime - Must restore from Backup | Downtime - Must wait for outage to end | |
| Single Gitaly Node + Geo Secondary | Downtime - Must restore from backup, can perform a manual failover to secondary | Downtime - Must restore from Backup, errors could have propagated to secondary | Manual intervention - failover to Geo secondary | |
| Sharded Gitaly Install | Partial Downtime - Only repos on impacted node affected, must restore from backup | Partial Downtime - Only repos on impacted node affected, must restore from backup | Downtime - Must wait for outage to end | |
| Sharded Gitaly Install + Geo Secondary | Partial Downtime - Only repos on impacted node affected, must restore from backup, could perform manual failover to secondary for impacted repos | Partial Downtime - Only repos on impacted node affected, must restore from backup, errors could have propagated to secondary | Manual intervention - failover to Geo secondary | |
| Gitaly Cluster Install* | No Downtime - will swap repository primary to another node after 10 seconds | N/A - All writes are voted on by multiple Gitaly Cluster nodes | Downtime - Must wait for outage to end | Snapshot backups for Gitaly Cluster nodes not supported at this time |
| Gitaly Cluster Install* + Geo Secondary | No Downtime - will swap repository primary to another node after 10 seconds | N/A - All writes are voted on by multiple Gitaly Cluster nodes | Manual intervention - failover to Geo secondary | Snapshot backups for Gitaly Cluster nodes not supported at this time |

2
doc/administration/gitaly/faq.md
View File

@ -35,7 +35,7 @@ For more information, see:
## Are there instructions for migrating to Gitaly Cluster?
Yes! For more information, see [Migrate to Gitaly Cluster](index.md#migrate-to-gitaly-cluster).
Yes! For more information, see [Migrating to Gitaly Cluster](index.md#migrating-to-gitaly-cluster).
## What are some repository storage recommendations?

111
doc/administration/gitaly/index.md
View File

@ -43,33 +43,49 @@ repository storage is either:
- Read requests are distributed between multiple Gitaly nodes, which can improve performance.
- Write requests are broadcast to repository replicas.
WARNING:
Engineering support for NFS for Git repositories is deprecated. Read the
[deprecation notice](#nfs-deprecation-notice).
## Guidance regarding Gitaly Cluster
## Recommendations
Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Please review existing technical limitations and considerations prior to deploying Gitaly Cluster.
The following table provides recommendations based on your requirements. Users means concurrent
users actively performing simultaneous Git operations.
- [Known issues](#known-issues)
- [Snapshot limitations](#snapshot-backup-and-recovery-limitations).
| User scaling | Reference architecture to use | GitLab instance configuration | Gitaly configuration | Git repository storage | Instances dedicated to Gitaly services |
|:-----------------------------------------------------------------------|:----------------------------------------------------------------|:---------------------------------------------|:--------------------------------|:-----------------------------------|:---------------------------------------|
| Up to 1000 users | [1K](../reference_architectures/1k_users.md) | Single instance for all of GitLab | Already integrated as a service | Local disk | 0 |
| Up to 2999 users | [2K](../reference_architectures/2k_users.md) | Horizontally-scaled GitLab instance (non-HA) | Single Gitaly server | Local disk of Gitaly instance | 1 |
| 3000 users and over | [3K](../reference_architectures/3k_users.md) | Horizontally-scaled GitLab instance with HA | Gitaly Cluster | Local disk of each Gitaly instance | 8 |
| RTO/RPO requirements for AZ failure tolerance regardless of user scale | [3K (with downscaling)](../reference_architectures/3k_users.md) | Custom (1) | Gitaly Cluster | Local disk of each Gitaly instance | 8 |
Please also review the [configuration guidance](configure_gitaly.md) and [Repository storage options](../repository_storage_paths.md) to make sure that Gitaly Cluster is the best set-up for you. Finally, refer to the following guidance:
1. If you need AZ failure tolerance for user scaling lower than 3K, please contact Customer Success
to discuss your RTO and RPO needs and what options exist to meet those objectives.
- If you have not yet migrated to Gitaly Cluster and want to continue using NFS, remain on the
service you are using. NFS is supported in 14.x releases.
- If you have not yet migrated to Gitaly Cluster but want to migrate away from NFS, you have two options - a sharded Gitaly instance or Gitaly Cluster.
- If you have migrated to Gitaly Cluster and the limitations and tradeoffs are not suitable for your environment, your options are:
1. [Migrate off Gitaly Cluster](#migrate-off-gitaly-cluster) back to your NFS solution
1. [Migrate off Gitaly Cluster](#migrate-off-gitaly-cluster) to NFS solution or to a sharded Gitaly instance.
GitLab [reference architectures](../reference_architectures/index.md) provide guidance for what
Gitaly configuration is recommended at each scale. The Gitaly configuration is noted by the
architecture diagrams and the table of required resources.
Reach out to your Technical Account Manager or customer support if you have any questions.
WARNING:
Some [known database inconsistency issues](#known-issues) exist in Gitaly Cluster. We recommend you
remain on your current service for now. We can adjust the date for
[NFS support removal](#nfs-deprecation-notice) if this applies to you.
### Known issues
The following table outlines current known issues impacting the use of Gitaly Cluster. For
the current status of these issues, please refer to the referenced issues and epics.
| Issue | Summary | How to avoid |
|:--------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------|
| Gitaly Cluster + Geo - Issues retrying failed syncs | If Gitaly Cluster is used on a Geo secondary site, repositories that have failed to sync could continue to fail when Geo tries to resync them. Recovering from this state requires assistance from support to run manual steps. Work is in-progress to update Gitaly Cluster to [identify repositories with a unique and persistent identifier](https://gitlab.com/gitlab-org/gitaly/-/issues/3485), which is expected to resolve the issue. | No known solution at this time. |
| Database inconsistencies due to repository access outside of Gitaly Cluster's control | Operations that write to the repository storage that do not go through normal Gitaly Cluster methods can cause database inconsistencies. These can include (but are not limited to) snapshot restoration for cluster node disks, node upgrades which modify files under Git control, or any other disk operation that may touch repository storage external to GitLab. The Gitaly team is actively working to provide manual commands to [reconcile the Praefect database with the repository storage](https://gitlab.com/groups/gitlab-org/-/epics/6723). | Don't directly change repositories on any Gitaly Cluster node at this time. |
| Praefect unable to insert data into the database due to migrations not being applied after an upgrade | If the database is not kept up to date with completed migrations, then the Praefect node is unable to perform normal operation. | Make sure the Praefect database is up and running with all migrations completed (For example: `/opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml sql-migrate-status` should show a list of all applied migrations). Consider [requesting live upgrade assistance](https://about.gitlab.com/support/scheduling-live-upgrade-assistance.html) so your upgrade plan can be reviewed by support. |
| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind will result in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you need to restore from backup, it's best to snapshot all Gitaly Cluster nodes at the same time and take a database dump of the Praefect database. |
### Snapshot backup and recovery limitations
Gitaly Cluster does not support snapshot backups because these can cause issues where the Praefect
database becomes out of sync with the disk storage. Because of how Praefect rebuilds the replication
metadata of Gitaly disk information during a restore, we recommend using the
[official backup and restore Rake tasks](../../raketasks/backup_restore.md). If you are unable to use this method, please contact customer support for restoration help.
To track progress on work on a solution for manually re-synchronizing the Praefect database with
disk storage, see [this epic](https://gitlab.com/groups/gitlab-org/-/epics/6575).
### What to do if you are on Gitaly Cluster experiencing an issue or limitation
Please contact customer support for immediate help in restoration or recovery.
## Gitaly
@ -188,26 +204,6 @@ WARNING:
If complete cluster failure occurs, disaster recovery plans should be executed. These can affect the
RPO and RTO discussed above.
### Known issues
The following table outlines current known issues impacting the use of Gitaly Cluster. For
the current status of these issues, please refer to the referenced issues and epics.
| Issue | Summary |
|:--------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Gitaly Cluster + Geo can cause database inconsistencies | There are some conditions during Geo replication that can cause database inconsistencies with Gitaly Cluster. These have been identified and are being resolved by updating Gitaly Cluster to [identify repositories with a unique and persistent identifier](https://gitlab.com/gitlab-org/gitaly/-/issues/3485). |
| Database inconsistencies due to repository access outside of Gitaly Cluster's control | Operations that write to the repository storage that do not go through normal Gitaly Cluster methods can cause database inconsistencies. These can include (but are not limited to) snapshot restoration for cluster node disks, node upgrades which modify files under Git control, or any other disk operation that may touch repository storage external to GitLab. The Gitaly team is actively working to provide manual commands to [reconcile the Praefect database with the repository storage](https://gitlab.com/groups/gitlab-org/-/epics/6723). |
### Snapshot backup and recovery limitations
Gitaly Cluster does not support snapshot backups because these can cause issues where the Praefect
database becomes out of sync with the disk storage. Because of how Praefect rebuilds the replication
metadata of Gitaly disk information during a restore, we recommend using the
[official backup and restore Rake tasks](../../raketasks/backup_restore.md).
To track progress on work on a solution for manually re-synchronizing the Praefect database with
disk storage, see [this epic](https://gitlab.com/groups/gitlab-org/-/epics/6575).
### Virtual storage
Virtual storage makes it viable to have a single repository storage in GitLab to simplify repository
@ -236,9 +232,7 @@ As with normal Gitaly storages, virtual storages can be sharded.
### Moving beyond NFS
WARNING:
Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
unavailable from GitLab 15.0. No further enhancements are planned for this feature.
Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be unavailable starting GitLab 15.0. Please see our [statement of support](https://about.gitlab.com/support/statement-of-support.html#gitaly-and-nfs) for more details.
[Network File System (NFS)](https://en.wikipedia.org/wiki/Network_File_System)
is not well suited to Git workloads which are CPU and IOPS sensitive.
@ -359,13 +353,9 @@ For configuration information, see [Configure replication factor](praefect.md#co
For more information on configuring Gitaly Cluster, see [Configure Gitaly Cluster](praefect.md).
## Migrate to Gitaly Cluster
## Migrating to Gitaly Cluster
We recommend you migrate to Gitaly Cluster if your [requirements recommend](#recommendations) Gitaly
Cluster.
Whether migrating to Gitaly Cluster because of [NFS support deprecation](index.md#nfs-deprecation-notice)
or to move from single Gitaly nodes, the basic process involves:
Please see [current guidance on Gitaly Cluster](#guidance-regarding-gitaly-cluster). The basic process for migrating to Gitaly Cluster involves:
1. Create the required storage. Refer to
[repository storage recommendations](faq.md#what-are-some-repository-storage-recommendations).
@ -376,8 +366,7 @@ or to move from single Gitaly nodes, the basic process involves:
WARNING:
Some [known database inconsistency issues](#known-issues) exist in Gitaly Cluster. We recommend you
remain on your current service for now. We can adjust the date for
[NFS support removal](#nfs-deprecation-notice) if this applies to you.
remain on your current service for now.
### Migrate off Gitaly Cluster
@ -631,20 +620,4 @@ The second facet presents the only real solution. For this, we developed
## NFS deprecation notice
Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
unavailable from GitLab 15.0. No further enhancements are planned for this feature.
Additional information:
- [Recommended NFS mount options and known issues with Gitaly and NFS](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
- [GitLab statement of support](https://about.gitlab.com/support/statement-of-support.html#gitaly-and-nfs).
GitLab recommends:
- Creating a [Gitaly Cluster](#gitaly-cluster) as soon as possible.
- [Moving your repositories](#migrate-to-gitaly-cluster) from NFS-based storage to Gitaly
Cluster.
We welcome your feedback on this process. You can:
- Raise a support ticket.
- [Comment on the epic](https://gitlab.com/groups/gitlab-org/-/epics/4916).
unavailable from GitLab 15.0. For further information, please see our [NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation) documentation.

4
doc/administration/gitaly/praefect.md
View File

@ -429,7 +429,7 @@ On the **Praefect** node:
WARNING:
If you have data on an already existing storage called
`default`, you should configure the virtual storage with another name and
[migrate the data to the Gitaly Cluster storage](index.md#migrate-to-gitaly-cluster)
[migrate the data to the Gitaly Cluster storage](index.md#migrating-to-gitaly-cluster)
afterwards.
Replace `PRAEFECT_INTERNAL_TOKEN` with a strong secret, which is used by
@ -893,7 +893,7 @@ Particular attention should be shown to:
WARNING:
If you have existing data stored on the default Gitaly storage,
you should [migrate the data your Gitaly Cluster storage](index.md#migrate-to-gitaly-cluster)
you should [migrate the data your Gitaly Cluster storage](index.md#migrating-to-gitaly-cluster)
first.
```ruby

40
doc/administration/nfs.md
View File

@ -20,12 +20,46 @@ file system performance, see
## Gitaly and NFS deprecation
Starting with GitLab version 14.0, support for NFS to store Git repository data will be deprecated. Technical customer support and engineering support will be available for the 14.x releases. Engineering will fix bugs and security vulnerabilities consistent with our [release and maintenance policy](../policy/maintenance.md#security-releases).
At the end of the 14.12 milestone (tenatively June 22nd, 2022) technical and engineering support for using NFS to store Git repository data will be officially at end-of-life. There will be no product changes or troubleshooting provided via Engineering, Security or Paid Support channels.
For those customers still running earlier versions of GitLab, [our support eligibility and maintenance policy applies](https://about.gitlab.com/support/statement-of-support.html#version-support).
For the 14.x releases, we will continue to help with Git related tickets from customers running one or more Gitaly servers with its data stored on NFS. Examples may include:
- Performance issues or timeouts accessing Git data
- Commits or branches vanish
- GitLab intermittently returns the wrong Git data (such as reporting that a repository has no branches)
Assistance will be limited to activities like:
- Verifying developers' workflow uses features like protected branches
- Reviewing GitLab event data from the database to advise if it looks like a force push over-wrote branches
- Verifying that NFS client mount options match our [documented recommendations](#mount-options)
- Analyzing the GitLab Workhorse and Rails logs, and determining that `500` errors being seen in the environment are caused by slow responses from Gitaly
GitLab support will be unable to continue with the investigation if:
- The date of the request is on or after the release of GitLab version 15.0, and
- Support Engineers and Management determine that all reasonable non-NFS root causes have been exhausted
If the issue is reproducible, or if it happens intermittently but regularly, GitLab Support will investigate providing the issue reproduces without the use of NFS. In order to reproduce without NFS, the affected repositories should be migrated to a different Gitaly shard, such as Gitaly cluster or a standalone Gitaly VM, backed with block storage.
### Why remove NFS for Git repository data
{:.no-toc}
NFS is not well-suited to a workload consisting of many small files, like Git repositories. NFS does provide a number of configuration options designed to improve performance. However, over time, a number of these mount options have proven to result in inconsistencies across multiple nodes mounting the NFS volume, up to and including data loss. Addressing these inconsistencies consume extraordinary development and support engineer time that hamper our ability to develop [Gitaly Cluster](gitaly/praefect.md), our purpose-built solution to addressing the deficiencies of NFS in this environment.
Please note that Gitaly Cluster provides highly-available Git repository storage. If this is not a requirement, single-node Gitaly backed by block storage is a suitable substitute.
Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
unavailable from GitLab 15.0. No further enhancements are planned for this feature.
Read:
- The [Gitaly and NFS deprecation notice](gitaly/index.md#nfs-deprecation-notice).
- [Moving beyond NFS](gitaly/index.md#moving-beyond-nfs).
- About the [correct mount options to use](#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
## Known kernel version incompatibilities
@ -370,8 +404,8 @@ sudo ufw allow from <client_ip_address> to any port nfs
### Upgrade to Gitaly Cluster or disable caching if experiencing data loss
WARNING:
Engineering support for NFS for Git repositories is deprecated. Read the
[Gitaly and NFS deprecation notice](gitaly/index.md#nfs-deprecation-notice).
Engineering support for NFS for Git repositories is deprecated. Read about
[moving beyond NFS](gitaly/index.md#moving-beyond-nfs).
Customers and users have reported data loss on high-traffic repositories when using NFS for Git repositories.
For example, we have seen:

2
doc/administration/operations/moving_repositories.md
View File

@ -27,7 +27,7 @@ For more information, see:
querying and scheduling snippet repository moves.
- [The API documentation](../../api/group_repository_storage_moves.md) details the endpoints for
querying and scheduling group repository moves **(PREMIUM SELF)**.
- [Migrate to Gitaly Cluster](../gitaly/index.md#migrate-to-gitaly-cluster).
- [Migrating to Gitaly Cluster](../gitaly/index.md#migrating-to-gitaly-cluster).
### Move Repositories

13
doc/administration/packages/container_registry.md
View File

@ -151,6 +151,19 @@ otherwise conflicts occur.
If your certificate provider provides the CA Bundle certificates, append them to the TLS certificate file.
An administrator may want the container registry listening on an arbitrary port such as `5678`.
However, the registry and application server are behind an AWS application load balancer that only
listens on ports `80` and `443`. The admin may simply remove the port number for
`registry_external_url`, so HTTP or HTTPS is assumed. Then, the rules apply that map the load
balancer to the registry from ports `80` or `443` to the arbitrary port. This is important if users
rely on the `docker login` example in the container registry. Here's an example:
```ruby
registry_external_url 'https://registry-gitlab.example.com'
registry_nginx['redirect_http_to_https'] = true
registry_nginx['listen_port'] = 5678
```
**Installations from source**
1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and

6
doc/administration/redis/replication_and_failover.md
View File

@ -647,6 +647,7 @@ persistence classes.
| `shared_state` | Store session-related and other persistent data. |
| `actioncable` | Pub/Sub queue backend for ActionCable. |
| `trace_chunks` | Store [CI trace chunks](../job_logs.md#enable-or-disable-incremental-logging) data. |
| `rate_limiting` | Store [rate limiting](../../user/admin_area/settings/user_and_ip_rate_limits.md) state. |
To make this work with Sentinel:
@ -659,6 +660,7 @@ To make this work with Sentinel:
gitlab_rails['redis_shared_state_instance'] = REDIS_SHARED_STATE_URL
gitlab_rails['redis_actioncable_instance'] = REDIS_ACTIONCABLE_URL
gitlab_rails['redis_trace_chunks_instance'] = REDIS_TRACE_CHUNKS_URL
gitlab_rails['redis_rate_limiting_instance'] = REDIS_RATE_LIMITING_URL
# Configure the Sentinels
gitlab_rails['redis_cache_sentinels'] = [
@ -681,6 +683,10 @@ To make this work with Sentinel:
{ host: TRACE_CHUNKS_SENTINEL_HOST, port: 26379 },
{ host: TRACE_CHUNKS_SENTINEL_HOST2, port: 26379 }
]
gitlab_rails['redis_rate_limiting_sentinels'] = [
{ host: RATE_LIMITING_SENTINEL_HOST, port: 26379 },
{ host: RATE_LIMITING_SENTINEL_HOST2, port: 26379 }
]
```
- Redis URLs should be in the format: `redis://:PASSWORD@SENTINEL_PRIMARY_NAME`, where:

2
doc/administration/reference_architectures/10k_users.md
View File

@ -2118,7 +2118,7 @@ unavailable from GitLab 15.0. No further enhancements are planned for this featu
Read:
- The [Gitaly and NFS deprecation notice](../gitaly/index.md#nfs-deprecation-notice).
- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
<div align="right">

2
doc/administration/reference_architectures/25k_users.md
View File

@ -2124,7 +2124,7 @@ unavailable from GitLab 15.0. No further enhancements are planned for this featu
Read:
- The [Gitaly and NFS deprecation notice](../gitaly/index.md#nfs-deprecation-notice).
- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
## Cloud Native Hybrid reference architecture with Helm Charts (alternative)

2
doc/administration/reference_architectures/2k_users.md
View File

@ -965,7 +965,7 @@ unavailable from GitLab 15.0. No further enhancements are planned for this featu
Read:
- The [Gitaly and NFS deprecation notice](../gitaly/index.md#nfs-deprecation-notice).
- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
## Cloud Native Hybrid reference architecture with Helm Charts (alternative)

2
doc/administration/reference_architectures/3k_users.md
View File

@ -2072,7 +2072,7 @@ unavailable from GitLab 15.0. No further enhancements are planned for this featu
Read:
- The [Gitaly and NFS deprecation notice](../gitaly/index.md#nfs-deprecation-notice).
- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
## Supported modifications for lower user counts (HA)

2
doc/administration/reference_architectures/50k_users.md
View File

@ -2138,7 +2138,7 @@ unavailable from GitLab 15.0. No further enhancements are planned for this featu
Read:
- The [Gitaly and NFS deprecation notice](../gitaly/index.md#nfs-deprecation-notice).
- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
## Cloud Native Hybrid reference architecture with Helm Charts (alternative)

2
doc/administration/reference_architectures/5k_users.md
View File

@ -2066,7 +2066,7 @@ unavailable from GitLab 15.0. No further enhancements are planned for this featu
Read:
- The [Gitaly and NFS deprecation notice](../gitaly/index.md#nfs-deprecation-notice).
- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
## Cloud Native Hybrid reference architecture with Helm Charts (alternative)

2
doc/administration/repository_storage_paths.md
View File

@ -174,4 +174,4 @@ information.
## Move repositories
To move a repository to a different repository storage (for example, from `default` to `storage2`), use the
same process as [migrating to Gitaly Cluster](gitaly/index.md#migrate-to-gitaly-cluster).
same process as [migrating to Gitaly Cluster](gitaly/index.md#migrating-to-gitaly-cluster).

3
doc/api/api_resources.md
View File

@ -42,6 +42,7 @@ The following API resources are available in the project context:
| [Events](events.md) | `/projects/:id/events` (also available for users and standalone) |
| [Feature Flags](feature_flags.md) | `/projects/:id/feature_flags` |
| [Feature Flag User Lists](feature_flag_user_lists.md) | `/projects/:id/feature_flags_user_lists` |
| [Integrations](integrations.md) | `/projects/:id/integrations` |
| [Invitations](invitations.md) | `/projects/:id/invitations` (also available for groups) |
| [Issues](issues.md) | `/projects/:id/issues` (also available for groups and standalone) |
| [Issues Statistics](issues_statistics.md) | `/projects/:id/issues_statistics` (also available for groups and standalone) |
@ -82,7 +83,7 @@ The following API resources are available in the project context:
| [Resource label events](resource_label_events.md) | `/projects/:id/issues/.../resource_label_events`, `/projects/:id/merge_requests/.../resource_label_events` (also available for groups) |
| [Runners](runners.md) | `/projects/:id/runners` (also available standalone) |
| [Search](search.md) | `/projects/:id/search` (also available for groups and standalone) |
| [Services](services.md) | `/projects/:id/services` |
| [Services](services.md) (renamed to [Integrations](integrations.md)) | `/projects/:id/services` |
| [Tags](tags.md) | `/projects/:id/repository/tags` |
| [User-starred metrics dashboards](metrics_user_starred_dashboards.md ) | `/projects/:id/metrics/user_starred_dashboards` |
| [Visual Review discussions](visual_review_discussions.md) **(PREMIUM)** | `/projects/:id/merge_requests/:merge_request_id/visual_review_discussions` |

10
doc/api/dependencies.md
View File

@ -11,6 +11,9 @@ This API is in an alpha stage and considered unstable.
The response payload may be subject to change or breakage
across GitLab releases.
> - Introduced in GitLab 12.1.
> - Pagination introduced in 14.4.
Every call to this endpoint requires authentication. To perform this call, user should be authorized to read repository.
To see vulnerabilities in response, user should be authorized to read
[Project Security Dashboard](../user/application_security/security_dashboard/index.md#project-security-dashboard).
@ -60,3 +63,10 @@ Example response:
}
]
```
## Dependencies pagination
By default, `GET` requests return 20 results at a time because the API results
are paginated.
Read more on [pagination](index.md#pagination).

1
doc/api/graphql/reference/index.md
View File

@ -16493,6 +16493,7 @@ Name of the feature that the callout is for.
| <a id="usercalloutfeaturenameenumregistration_enabled_callout"></a>`REGISTRATION_ENABLED_CALLOUT` | Callout feature name for registration_enabled_callout. |
| <a id="usercalloutfeaturenameenumsecurity_configuration_devops_alert"></a>`SECURITY_CONFIGURATION_DEVOPS_ALERT` | Callout feature name for security_configuration_devops_alert. |
| <a id="usercalloutfeaturenameenumsecurity_configuration_upgrade_banner"></a>`SECURITY_CONFIGURATION_UPGRADE_BANNER` | Callout feature name for security_configuration_upgrade_banner. |
| <a id="usercalloutfeaturenameenumsecurity_newsletter_callout"></a>`SECURITY_NEWSLETTER_CALLOUT` | Callout feature name for security_newsletter_callout. |
| <a id="usercalloutfeaturenameenumsuggest_pipeline"></a>`SUGGEST_PIPELINE` | Callout feature name for suggest_pipeline. |
| <a id="usercalloutfeaturenameenumsuggest_popover_dismissed"></a>`SUGGEST_POPOVER_DISMISSED` | Callout feature name for suggest_popover_dismissed. |
| <a id="usercalloutfeaturenameenumtabs_position_highlight"></a>`TABS_POSITION_HIGHLIGHT` | Callout feature name for tabs_position_highlight. |

2
doc/api/group_repository_storage_moves.md
View File

@ -10,7 +10,7 @@ type: reference
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/53016) in GitLab 13.9.
Group repositories can be moved between storages. This API can help you when
[migrating to Gitaly Cluster](../administration/gitaly/index.md#migrate-to-gitaly-cluster), for
[migrating to Gitaly Cluster](../administration/gitaly/index.md#migrating-to-gitaly-cluster), for
example, or to migrate a [group wiki](../user/project/wiki/index.md#group-wikis).
As group repository storage moves are processed, they transition through different states. Values

1473
doc/api/integrations.md Normal file
View File

File diff suppressed because it is too large Load Diff

2
doc/api/project_repository_storage_moves.md
View File

@ -10,7 +10,7 @@ type: reference
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31285) in GitLab 13.0.
Project repositories including wiki and design repositories can be moved between storages. This can be useful when
[migrating to Gitaly Cluster](../administration/gitaly/index.md#migrate-to-gitaly-cluster),
[migrating to Gitaly Cluster](../administration/gitaly/index.md#migrating-to-gitaly-cluster),
for example.
As project repository storage moves are processed, they transition through different states. Values

1472
doc/api/services.md
View File

File diff suppressed because it is too large Load Diff

2
doc/api/snippet_repository_storage_moves.md
View File

@ -10,7 +10,7 @@ type: reference
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49228) in GitLab 13.8.
Snippet repositories can be moved between storages. This can be useful when
[migrating to Gitaly Cluster](../administration/gitaly/index.md#migrate-to-gitaly-cluster), for
[migrating to Gitaly Cluster](../administration/gitaly/index.md#migrating-to-gitaly-cluster), for
example.
As snippet repository storage moves are processed, they transition through different states. Values

2
doc/api/usage_data.md
View File

@ -13,7 +13,7 @@ The Service Data API is associated with [Service Ping](../development/service_pi
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/57270) in GitLab 13.11.
Export all metric definitions as a single YAML file, similar to the [Metrics Dictionary](https://gitlab-org.gitlab.io/growth/product-intelligence/metric-dictionary), for easier importing.
Export all metric definitions as a single YAML file, similar to the [Metrics Dictionary](https://metrics.gitlab.com/index.html), for easier importing.
```plaintext
GET /usage_data/metric_definitions

4
doc/api/users.md
View File

@ -259,7 +259,7 @@ GET /users?with_custom_attributes=true
## Single user
Get a single user. This endpoint can be accessed without authentication.
Get a single user.
### For user
@ -806,7 +806,7 @@ Example response:
### Followers and following
Get the followers of a user. This endpoint can be accessed without authentication.
Get the followers of a user.
```plaintext
GET /users/:id/followers

6
doc/development/fe_guide/storybook.md
View File

@ -49,3 +49,9 @@ To add a story:
- Specify the `title` field of the story as the component's file path from the `javascripts/` directory,
e.g. if the component is located at `app/assets/javascripts/vue_shared/components/sidebar/todo_toggle/todo_button.vue`, specify the `title` as
`vue_shared/components/To-do Button`. This will ensure the Storybook navigation maps closely to our internal directory structure.
## Mock backend APIs
GitLab’s Storybook uses [MirajeJS](https://miragejs.com/) to mock REST and GraphQL APIs. Storybook shares the MirajeJS server
with the [frontend integration tests](../testing_guide/testing_levels.md#frontend-integration-tests). You can find the MirajeJS
configuration files in `spec/frontend_integration/mock_server`.

2
doc/development/redis.md
View File

@ -15,6 +15,8 @@ GitLab uses [Redis](https://redis.io) for the following distinct purposes:
- To manage the shared application state.
- To store CI trace chunks.
- As a Pub/Sub queue backend for ActionCable.
- CI trace chunk storage.
- Rate limiting state storage.
In most environments (including the GDK), all of these point to the same
Redis instance.

2
doc/development/service_ping/dictionary.md
View File

@ -1,4 +1,4 @@
---
redirect_to: 'https://gitlab-org.gitlab.io/growth/product-intelligence/metric-dictionary'
redirect_to: 'https://metrics.gitlab.com/index.html'
remove_date: '2021-11-10'
---

2
doc/development/service_ping/metrics_dictionary.md
View File

@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Metrics Dictionary Guide
[Service Ping](index.md) metrics are defined in the
[Metrics Dictionary](https://gitlab-org.gitlab.io/growth/product-intelligence/metric-dictionary).
[Metrics Dictionary](https://metrics.gitlab.com/index.html).
This guide describes the dictionary and how it's implemented.
## Metrics Definition and validation

2
doc/development/service_ping/review_guidelines.md
View File

@ -14,7 +14,7 @@ general best practices for code reviews, refer to our [code review guide](../cod
## Resources for reviewers
- [Service Ping Guide](index.md)
- [Metrics Dictionary](https://gitlab-org.gitlab.io/growth/product-intelligence/metric-dictionary)
- [Metrics Dictionary](https://metrics.gitlab.com/index.html)
## Review process

2
doc/development/usage_ping/dictionary.md
View File

@ -1,4 +1,4 @@
---
redirect_to: 'https://gitlab-org.gitlab.io/growth/product-intelligence/metric-dictionary'
redirect_to: 'https://metrics.gitlab.com/index.html'
remove_date: '2021-11-10'
---

6
doc/user/profile/account/two_factor_authentication.md
View File

@ -75,6 +75,7 @@ To enable 2FA:
1. **In GitLab:**
1. Enter the six-digit pin number from the entry on your device into the **Pin
code** field.
1. Enter your current password.
1. Select **Submit**.
If the pin you entered was correct, a message displays indicating that
@ -365,7 +366,8 @@ If you ever need to disable 2FA:
1. Sign in to your GitLab account.
1. Go to your [**User settings**](../index.md#access-your-user-settings).
1. Go to **Account**.
1. Click **Disable**, under **Two-Factor Authentication**.
1. Select **Manage two-factor authentication**.
1. Under **Two-Factor Authentication**, enter your current password and select **Disable**.
This clears all your two-factor authentication registrations, including mobile
applications and U2F / WebAuthn devices.
@ -460,7 +462,7 @@ To regenerate 2FA recovery codes, you need access to a desktop browser:
1. Go to your [**User settings**](../index.md#access-your-user-settings).
1. Select **Account > Two-Factor Authentication (2FA)**.
1. If you've already configured 2FA, click **Manage two-factor authentication**.
1. In the **Register Two-Factor Authenticator** pane, click **Regenerate recovery codes**.
1. In the **Register Two-Factor Authenticator** pane, enter your current password and select **Regenerate recovery codes**.
NOTE:
If you regenerate 2FA recovery codes, save them. You can't use any previously created 2FA codes.

6
doc/user/project/integrations/asana.md
View File

@ -4,9 +4,9 @@ group: Integrations
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Asana service **(FREE)**
# Asana integration **(FREE)**
This service adds commit messages as comments to Asana tasks.
This integration adds commit messages as comments to Asana tasks.
Once enabled, commit messages are checked for Asana task URLs (for example,
`https://app.asana.com/0/123456/987654`) or task IDs starting with `#`
(for example, `#987654`). Every task ID found gets the commit comment added to it.
@ -23,7 +23,7 @@ You can use either of these words:
- `closed`
- `closing`
See also the [Asana service API documentation](../../../api/services.md#asana).
See also the [Asana integration API documentation](../../../api/integrations.md#asana).
## Setup

4
doc/user/project/integrations/overview.md
View File

@ -32,7 +32,7 @@ Click on the service links to see further configuration instructions and details
| [Bugzilla](bugzilla.md) | Use Bugzilla as the issue tracker. | **{dotted-circle}** No |
| Buildkite | Run CI/CD pipelines with Buildkite. | **{check-circle}** Yes |
| Campfire | Connect to chat. | **{dotted-circle}** No |
| [Confluence Workspace](../../../api/services.md#confluence-service) | Replace the link to the internal wiki with a link to a Confluence Cloud Workspace. | **{dotted-circle}** No |
| [Confluence Workspace](../../../api/integrations.md#confluence-integration) | Replace the link to the internal wiki with a link to a Confluence Cloud Workspace. | **{dotted-circle}** No |
| [Custom issue tracker](custom_issue_tracker.md) | Use a custom issue tracker. | **{dotted-circle}** No |
| [Datadog](../../../integration/datadog.md) | Trace your GitLab pipelines with Datadog. | **{check-circle}** Yes |
| [Discord Notifications](discord_notifications.md) | Send notifications about project events to a Discord channel. | **{dotted-circle}** No |
@ -40,7 +40,7 @@ Click on the service links to see further configuration instructions and details
| [Emails on push](emails_on_push.md) | Send commits and diff of each push by email. | **{dotted-circle}** No |
| [EWM](ewm.md) | Use IBM Engineering Workflow Management as the issue tracker. | **{dotted-circle}** No |
| [External wiki](../wiki/index.md#link-an-external-wiki) | Link an external wiki. | **{dotted-circle}** No |
| [Flowdock](../../../api/services.md#flowdock) | Send notifications from GitLab to Flowdock flows. | **{dotted-circle}** No |
| [Flowdock](../../../api/integrations.md#flowdock) | Send notifications from GitLab to Flowdock flows. | **{dotted-circle}** No |
| [GitHub](github.md) | Obtain statuses for commits and pull requests. | **{dotted-circle}** No |
| [Google Chat](hangouts_chat.md) | Send notifications from your GitLab project to a room in Google Chat.| **{dotted-circle}** No |
| [irker (IRC gateway)](irker.md) | Send IRC messages. | **{dotted-circle}** No |

2
doc/user/project/integrations/pivotal_tracker.md
View File

@ -29,7 +29,7 @@ Read more about the
[Source Commits endpoint](https://www.pivotaltracker.com/help/api/rest/v5#Source_Commits) in
the Pivotal Tracker API documentation.
See also the [Pivotal Tracker service API documentation](../../../api/services.md#pivotal-tracker).
See also the [Pivotal Tracker integration API documentation](../../../api/integrations.md#pivotal-tracker).
## Set up Pivotal Tracker

4
doc/user/project/settings/import_export.md
View File

@ -135,9 +135,11 @@ The following items are **not** exported:
- Build traces and artifacts
- Container registry images
- CI/CD variables
- Pipeline triggers
- Webhooks
- Any encrypted tokens
- Merge Request Approvers
- Repository size limits
These content rules also apply to creating projects from templates on the
[group](../../group/custom_project_templates.md)
@ -261,7 +263,7 @@ reduce the repository size for another import attempt.
git gc --prune=now --aggressive
# Prepare recreating an importable file
git bundle create ../project.bundle smaller-tmp-main
git bundle create ../project.bundle <default-branch-name>
cd ..
mv project/ ../"$EXPORT"-project
cd ..

2
lib/api/api.rb
View File

@ -260,7 +260,7 @@ module API
mount ::API::ResourceAccessTokens
mount ::API::RubygemPackages
mount ::API::Search
mount ::API::Services
mount ::API::Integrations
mount ::API::Settings
mount ::API::SidekiqMetrics
mount ::API::SnippetRepositoryStorageMoves

2
lib/api/helpers/integrations_helpers.rb
View File

@ -2,7 +2,7 @@
module API
module Helpers
# Helpers module for API::Services
# Helpers module for API::Integrations
#
# The data structures inside this model are returned using class methods,
# allowing EE to extend them where necessary.

4
lib/api/import_bitbucket_server.rb
View File

@ -4,6 +4,10 @@ module API
class ImportBitbucketServer < ::API::Base
feature_category :importers
before do
forbidden! unless Gitlab::CurrentSettings.import_sources&.include?('bitbucket_server')
end
helpers do
def client
@client ||= BitbucketServer::Client.new(credentials)

178
lib/api/integrations.rb Normal file
View File

@ -0,0 +1,178 @@
# frozen_string_literal: true
module API
class Integrations < ::API::Base
feature_category :integrations
integrations = Helpers::IntegrationsHelpers.integrations
integration_classes = Helpers::IntegrationsHelpers.integration_classes
if Rails.env.development?
integrations['mock-ci'] = [
{
required: true,
name: :mock_service_url,
type: String,
desc: 'URL to the mock integration'
}
]
integrations['mock-deployment'] = []
integrations['mock-monitoring'] = []
integration_classes += Helpers::IntegrationsHelpers.development_integration_classes
end
INTEGRATIONS = integrations.freeze
integration_classes.each do |integration|
event_names = integration.try(:event_names) || next
event_names.each do |event_name|
INTEGRATIONS[integration.to_param.tr("_", "-")] << {
required: false,
name: event_name.to_sym,
type: String,
desc: IntegrationsHelper.integration_event_description(integration, event_name)
}
end
end
TRIGGER_INTEGRATIONS = {
'mattermost-slash-commands' => [
{
name: :token,
type: String,
desc: 'The Mattermost token'
}
],
'slack-slash-commands' => [
{
name: :token,
type: String,
desc: 'The Slack token'
}
]
}.freeze
helpers do
def integration_attributes(integration)
integration.fields.inject([]) do |arr, hash|
arr << hash[:name].to_sym
end
end
end
# The API officially documents only the `:id/integrations` API paths.
# We support the older `id:/services` path for backwards-compatibility in API V4.
# The support for `:id/services` can be dropped if we create an API V5.
[':id/services', ':id/integrations'].each do |path|
params do
requires :id, type: String, desc: 'The ID of a project'
end
resource :projects, requirements: API::NAMESPACE_OR_PROJECT_REQUIREMENTS do
before { authenticate! }
before { authorize_admin_project }
desc 'Get all active project integrations' do
success Entities::ProjectIntegrationBasic
end
get path do
integrations = user_project.integrations.active
present integrations, with: Entities::ProjectIntegrationBasic
end
INTEGRATIONS.each do |slug, settings|
desc "Set #{slug} integration for project"
params do
settings.each do |setting|
if setting[:required]
requires setting[:name], type: setting[:type], desc: setting[:desc]
else
optional setting[:name], type: setting[:type], desc: setting[:desc]
end
end
end
put "#{path}/#{slug}" do
integration = user_project.find_or_initialize_integration(slug.underscore)
params = declared_params(include_missing: false).merge(active: true)
if integration.update(params)
present integration, with: Entities::ProjectIntegration
else
render_api_error!('400 Bad Request', 400)
end
end
end
desc "Delete an integration from a project"
params do
requires :slug, type: String, values: INTEGRATIONS.keys, desc: 'The name of the integration'
end
delete "#{path}/:slug" do
integration = user_project.find_or_initialize_integration(params[:slug].underscore)
destroy_conditionally!(integration) do
attrs = integration_attributes(integration).index_with { nil }.merge(active: false)
render_api_error!('400 Bad Request', 400) unless integration.update(attrs)
end
end
desc 'Get the integration settings for a project' do
success Entities::ProjectIntegration
end
params do
requires :slug, type: String, values: INTEGRATIONS.keys, desc: 'The name of the integration'
end
get "#{path}/:slug" do
integration = user_project.find_or_initialize_integration(params[:slug].underscore)
not_found!('Integration') unless integration&.persisted?
present integration, with: Entities::ProjectIntegration
end
end
TRIGGER_INTEGRATIONS.each do |integration_slug, settings|
helpers do
def slash_command_integration(project, integration_slug, params)
project.integrations.active.find do |integration|
integration.try(:token) == params[:token] && integration.to_param == integration_slug.underscore
end
end
end
params do
requires :id, type: String, desc: 'The ID of a project'
end
resource :projects, requirements: API::NAMESPACE_OR_PROJECT_REQUIREMENTS do
desc "Trigger a slash command for #{integration_slug}" do
detail 'Added in GitLab 8.13'
end
params do
settings.each do |setting|
requires setting[:name], type: setting[:type], desc: setting[:desc]
end
end
post "#{path}/#{integration_slug.underscore}/trigger" do
project = find_project(params[:id])
# This is not accurate, but done to prevent leakage of the project names
not_found!('Integration') unless project
integration = slash_command_integration(project, integration_slug, params)
result = integration.try(:trigger, params)
if result
status result[:status] || 200
present result
else
not_found!('Integration')
end
end
end
end
end
end
end
API::Integrations.prepend_mod_with('API::Integrations')

2
lib/api/invitations.rb
View File

@ -46,6 +46,8 @@ module API
source = find_source(source_type, params[:id])
query = params[:query]
authorize_admin_source!(source_type, source)
invitations = paginate(retrieve_member_invitations(source, query))
present_member_invitations invitations

5
lib/api/projects.rb
View File

@ -89,6 +89,10 @@ module API
Gitlab::AppLogger.info({ message: "File exceeds maximum size", file_bytes: file.size, project_id: user_project.id, project_path: user_project.full_path, upload_allowed: allowed })
end
end
def check_import_by_url_is_enabled
forbidden! unless Gitlab::CurrentSettings.import_sources&.include?('git')
end
end
helpers do
@ -267,6 +271,7 @@ module API
attrs = declared_params(include_missing: false)
attrs = translate_params_for_compatibility(attrs)
filter_attributes_using_license!(attrs)
check_import_by_url_is_enabled if params[:import_url].present?
project = ::Projects::CreateService.new(current_user, attrs).execute
if project.saved?

173
lib/api/services.rb
View File

@ -1,173 +0,0 @@
# frozen_string_literal: true
module API
class Services < ::API::Base
feature_category :integrations
integrations = Helpers::IntegrationsHelpers.integrations
integration_classes = Helpers::IntegrationsHelpers.integration_classes
if Rails.env.development?
integrations['mock-ci'] = [
{
required: true,
name: :mock_service_url,
type: String,
desc: 'URL to the mock service'
}
]
integrations['mock-deployment'] = []
integrations['mock-monitoring'] = []
integration_classes += Helpers::IntegrationsHelpers.development_integration_classes
end
INTEGRATIONS = integrations.freeze
integration_classes.each do |integration|
event_names = integration.try(:event_names) || next
event_names.each do |event_name|
INTEGRATIONS[integration.to_param.tr("_", "-")] << {
required: false,
name: event_name.to_sym,
type: String,
desc: IntegrationsHelper.integration_event_description(integration, event_name)
}
end
end
TRIGGER_INTEGRATIONS = {
'mattermost-slash-commands' => [
{
name: :token,
type: String,
desc: 'The Mattermost token'
}
],
'slack-slash-commands' => [
{
name: :token,
type: String,
desc: 'The Slack token'
}
]
}.freeze
params do
requires :id, type: String, desc: 'The ID of a project'
end
resource :projects, requirements: API::NAMESPACE_OR_PROJECT_REQUIREMENTS do
before { authenticate! }
before { authorize_admin_project }
helpers do
def integration_attributes(integration)
integration.fields.inject([]) do |arr, hash|
arr << hash[:name].to_sym
end
end
end
desc 'Get all active project integrations' do
success Entities::ProjectIntegrationBasic
end
get ":id/services" do
integrations = user_project.integrations.active
present integrations, with: Entities::ProjectIntegrationBasic
end
INTEGRATIONS.each do |slug, settings|
desc "Set #{slug} integration for project"
params do
settings.each do |setting|
if setting[:required]
requires setting[:name], type: setting[:type], desc: setting[:desc]
else
optional setting[:name], type: setting[:type], desc: setting[:desc]
end
end
end
put ":id/services/#{slug}" do
integration = user_project.find_or_initialize_integration(slug.underscore)
params = declared_params(include_missing: false).merge(active: true)
if integration.update(params)
present integration, with: Entities::ProjectIntegration
else
render_api_error!('400 Bad Request', 400)
end
end
end
desc "Delete an integration from a project"
params do
requires :slug, type: String, values: INTEGRATIONS.keys, desc: 'The name of the service'
end
delete ":id/services/:slug" do
integration = user_project.find_or_initialize_integration(params[:slug].underscore)
destroy_conditionally!(integration) do
attrs = integration_attributes(integration).index_with { nil }.merge(active: false)
render_api_error!('400 Bad Request', 400) unless integration.update(attrs)
end
end
desc 'Get the integration settings for a project' do
success Entities::ProjectIntegration
end
params do
requires :slug, type: String, values: INTEGRATIONS.keys, desc: 'The name of the service'
end
get ":id/services/:slug" do
integration = user_project.find_or_initialize_integration(params[:slug].underscore)
not_found!('Service') unless integration&.persisted?
present integration, with: Entities::ProjectIntegration
end
end
TRIGGER_INTEGRATIONS.each do |integration_slug, settings|
helpers do
def slash_command_integration(project, integration_slug, params)
project.integrations.active.find do |integration|
integration.try(:token) == params[:token] && integration.to_param == integration_slug.underscore
end
end
end
params do
requires :id, type: String, desc: 'The ID of a project'
end
resource :projects, requirements: API::NAMESPACE_OR_PROJECT_REQUIREMENTS do
desc "Trigger a slash command for #{integration_slug}" do
detail 'Added in GitLab 8.13'
end
params do
settings.each do |setting|
requires setting[:name], type: setting[:type], desc: setting[:desc]
end
end
post ":id/services/#{integration_slug.underscore}/trigger" do
project = find_project(params[:id])
# This is not accurate, but done to prevent leakage of the project names
not_found!('Service') unless project
integration = slash_command_integration(project, integration_slug, params)
result = integration.try(:trigger, params)
if result
status result[:status] || 200
present result
else
not_found!('Service')
end
end
end
end
end
end
API::Services.prepend_mod_with('API::Services')

8
lib/api/users.rb
View File

@ -140,7 +140,10 @@ module API
end
# rubocop: disable CodeReuse/ActiveRecord
get ":id", feature_category: :users do
forbidden!('Not authorized!') unless current_user
user = User.find_by(id: params[:id])
not_found!('User') unless user && can?(current_user, :read_user, user)
opts = { with: current_user&.admin? ? Entities::UserDetailsWithAdmin : Entities::User, current_user: current_user }
@ -156,6 +159,7 @@ module API
end
get ":user_id/status", requirements: API::USER_REQUIREMENTS, feature_category: :users do
user = find_user(params[:user_id])
not_found!('User') unless user && can?(current_user, :read_user, user)
present user.status || {}, with: Entities::UserStatus
@ -203,6 +207,8 @@ module API
use :pagination
end
get ':id/following', feature_category: :users do
forbidden!('Not authorized!') unless current_user
user = find_user(params[:id])
not_found!('User') unless user && can?(current_user, :read_user_profile, user)
@ -217,6 +223,8 @@ module API
use :pagination
end
get ':id/followers', feature_category: :users do
forbidden!('Not authorized!') unless current_user
user = find_user(params[:id])
not_found!('User') unless user && can?(current_user, :read_user_profile, user)

21
lib/banzai/filter/spaced_link_filter.rb
View File

@ -26,14 +26,17 @@ module Banzai
# Pattern to match a standard markdown link
#
# Rubular: http://rubular.com/r/2EXEQ49rg5
LINK_OR_IMAGE_PATTERN = %r{
(?<preview_operator>!)?
\[(?<text>.+?)\]
\(
(?<new_link>.+?)
(?<title>\ ".+?")?
\)
}x.freeze
#
# This pattern is vulnerable to malicious inputs, so use Gitlab::UntrustedRegexp
# to place bounds on execution time
LINK_OR_IMAGE_PATTERN = Gitlab::UntrustedRegexp.new(
'(?P<preview_operator>!)?' \
'\[(?P<text>.+?)\]' \
'\(' \
'(?P<new_link>.+?)' \
'(?P<title>\ ".+?")?' \
'\)'
)
# Text matching LINK_OR_IMAGE_PATTERN inside these elements will not be linked
IGNORE_PARENTS = %w(a code kbd pre script style).to_set
@ -48,7 +51,7 @@ module Banzai
doc.xpath(TEXT_QUERY).each do |node|
content = node.to_html
next unless content.match(LINK_OR_IMAGE_PATTERN)
next unless LINK_OR_IMAGE_PATTERN.match(content)
html = spaced_link_filter(content)

6
lib/bulk_imports/ndjson_pipeline.rb
View File

@ -14,7 +14,7 @@ module BulkImports
relation_definition = import_export_config.top_relation_tree(relation)
deep_transform_relation!(relation_hash, relation, relation_definition) do |key, hash|
Gitlab::ImportExport::Group::RelationFactory.create(
relation_factory.create(
relation_index: relation_index,
relation_sym: key.to_sym,
relation_hash: hash,
@ -83,6 +83,10 @@ module BulkImports
"Gitlab::ImportExport::#{portable.class}::ObjectBuilder".constantize
end
def relation_factory
"Gitlab::ImportExport::#{portable.class}::RelationFactory".constantize
end
def relation
self.class.relation
end

15
lib/bulk_imports/projects/pipelines/issues_pipeline.rb Normal file
View File

@ -0,0 +1,15 @@
# frozen_string_literal: true
module BulkImports
module Projects
module Pipelines
class IssuesPipeline
include NdjsonPipeline
relation_name 'issues'
extractor ::BulkImports::Common::Extractors::NdjsonExtractor, relation: relation
end
end
end
end

6
lib/bulk_imports/projects/stage.rb
View File

@ -15,9 +15,13 @@ module BulkImports
pipeline: BulkImports::Common::Pipelines::LabelsPipeline,
stage: 1
},
issues: {
pipeline: BulkImports::Projects::Pipelines::IssuesPipeline,
stage: 2
},
finisher: {
pipeline: BulkImports::Common::Pipelines::EntityFinisher,
stage: 2
stage: 3
}
}
end

6
lib/gitlab/auth.rb
View File

@ -172,7 +172,11 @@ module Gitlab
user = find_with_user_password(login, password)
return unless user
raise Gitlab::Auth::MissingPersonalAccessTokenError if user.two_factor_enabled?
verifier = TwoFactorAuthVerifier.new(user)
if user.two_factor_enabled? || verifier.two_factor_authentication_enforced?
raise Gitlab::Auth::MissingPersonalAccessTokenError
end
Gitlab::Auth::Result.new(user, nil, :gitlab_or_ldap, full_authentication_abilities)
end

4
lib/gitlab/auth/auth_finders.rb
View File

@ -342,6 +342,10 @@ module Gitlab
Gitlab::PathRegex.repository_git_lfs_route_regex.match?(current_request.path)
end
def git_or_lfs_request?
git_request? || git_lfs_request?
end
def archive_request?
current_request.path.include?('/-/archive/')
end

24
lib/gitlab/auth/request_authenticator.rb
View File

@ -35,13 +35,31 @@ module Gitlab
find_user_from_static_object_token(request_format) ||
find_user_from_basic_auth_job ||
find_user_from_job_token ||
find_user_from_lfs_token ||
find_user_from_personal_access_token ||
find_user_from_basic_auth_password
find_user_from_personal_access_token_for_api_or_git ||
find_user_for_git_or_lfs_request
rescue Gitlab::Auth::AuthenticationError
nil
end
# To prevent Rack Attack from incorrectly rate limiting
# authenticated Git activity, we need to authenticate the user
# from other means (e.g. HTTP Basic Authentication) only if the
# request originated from a Git or Git LFS
# request. Repositories::GitHttpClientController or
# Repositories::LfsApiController normally does the authentication,
# but Rack Attack runs before those controllers.
def find_user_for_git_or_lfs_request
return unless git_or_lfs_request?
find_user_from_lfs_token || find_user_from_basic_auth_password
end
def find_user_from_personal_access_token_for_api_or_git
return unless api_request? || git_or_lfs_request?
find_user_from_personal_access_token
end
def valid_access_token?(scopes: [])
validate_access_token!(scopes: scopes)

4
lib/gitlab/auth/two_factor_auth_verifier.rb
View File

@ -9,6 +9,10 @@ module Gitlab
@current_user = current_user
end
def two_factor_authentication_enforced?
two_factor_authentication_required? && two_factor_grace_period_expired?
end
def two_factor_authentication_required?
Gitlab::CurrentSettings.require_two_factor_authentication? ||
current_user&.require_two_factor_authentication_from_group?

Some files were not shown because too many files have changed in this diff Show More