From d6085b68c5dcc2bd4dc884d2fa6b75831f85c883 Mon Sep 17 00:00:00 2001 From: GitLab Bot Date: Sat, 20 Nov 2021 00:14:01 +0000 Subject: [PATCH] Add latest changes from gitlab-org/gitlab@master --- GITALY_SERVER_VERSION | 2 +- doc/administration/compliance.md | 2 +- doc/api/groups.md | 4 +- doc/development/code_review.md | 2 +- doc/subscriptions/bronze_starter.md | 2 +- doc/user/group/index.md | 8 +- locale/gitlab.pot | 2 +- rubocop/cop/static_translation_definition.rb | 10 +- .../markdown_golden_master_examples.yml | 792 ++++++++++++++++++ .../markdown_processing_examples.js | 27 - .../markdown_processing_spec.js | 27 +- .../markdown_processing_spec_helper.js | 75 ++ spec/frontend/fixtures/api_markdown.rb | 65 -- spec/frontend/fixtures/api_markdown.yml | 289 ------- .../api/markdown_golden_master_spec.rb | 9 + .../cop/static_translation_definition_spec.rb | 16 +- spec/spec_helper.rb | 7 + .../markdown_golden_master_shared_examples.rb | 127 +++ 18 files changed, 1058 insertions(+), 408 deletions(-) create mode 100644 spec/fixtures/markdown/markdown_golden_master_examples.yml delete mode 100644 spec/frontend/content_editor/markdown_processing_examples.js create mode 100644 spec/frontend/content_editor/markdown_processing_spec_helper.js delete mode 100644 spec/frontend/fixtures/api_markdown.rb delete mode 100644 spec/frontend/fixtures/api_markdown.yml create mode 100644 spec/requests/api/markdown_golden_master_spec.rb create mode 100644 spec/support/shared_contexts/markdown_golden_master_shared_examples.rb diff --git a/GITALY_SERVER_VERSION b/GITALY_SERVER_VERSION index 447fd9d98b9..4c5360e4050 100644 --- a/GITALY_SERVER_VERSION +++ b/GITALY_SERVER_VERSION @@ -1 +1 @@ -b6dda5d1f7a7e05c34ed0f72f161a46aee536d75 \ No newline at end of file +6715e04b0a20714776227eaa67d68a6fa1ccfbdf diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index a05495c024e..7cecc0c30fd 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -73,7 +73,7 @@ These features can also help with compliance requirements: - [**Generate reports on permission levels of users**](../user/admin_area/index.md#user-permission-export) (for instances): Administrators can generate a report listing all users' access permissions for groups and projects in the instance. -- [**Lock project membership to group**](../user/group/index.md#prevent-members-from-being-added-to-a-group) (for +- [**Lock project membership to group**](../user/group/index.md#prevent-members-from-being-added-to-projects-in-a-group) (for groups): Group owners can prevent new members from being added to projects within a group. - [**LDAP group sync**](auth/ldap/ldap_synchronization.md#group-sync) (for instances): Gives administrators the ability to automatically sync groups and manage SSH keys, permissions, and authentication, so you can focus on building your diff --git a/doc/api/groups.md b/doc/api/groups.md index 5faa63585c1..dbe56dd9f17 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -788,7 +788,7 @@ Parameters: | `name` | string | yes | The name of the group. | | `path` | string | yes | The path of the group. | | `description` | string | no | The group's description. | -| `membership_lock` | boolean | no | **(PREMIUM)** Prevent adding new members to project membership within this group. | +| `membership_lock` | boolean | no | **(PREMIUM)** Prevent adding new members to projects within this group. | | `visibility` | string | no | The group's visibility. Can be `private`, `internal`, or `public`. | | `share_with_group_lock` | boolean | no | Prevent sharing a project with another group within this group. | | `require_two_factor_authentication` | boolean | no | Require all users in this group to setup Two-factor authentication. | @@ -864,7 +864,7 @@ PUT /groups/:id | `name` | string | no | The name of the group. | | `path` | string | no | The path of the group. | | `description` | string | no | The description of the group. | -| `membership_lock` | boolean | no | **(PREMIUM)** Prevent adding new members to project membership within this group. | +| `membership_lock` | boolean | no | **(PREMIUM)** Prevent adding new members to projects within this group. | | `share_with_group_lock` | boolean | no | Prevent sharing a project with another group within this group. | | `visibility` | string | no | The visibility level of the group. Can be `private`, `internal`, or `public`. | | `require_two_factor_authentication` | boolean | no | Require all users in this group to setup Two-factor authentication. | diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 7e797309a26..742d183e15c 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -381,7 +381,7 @@ first time. ### Requesting a review When you are ready to have your merge request reviewed, -you should request an initial review by assigning it to a reviewer from your group or team. +you should [request an initial review](../user/project/merge_requests/getting_started.md#reviewer) by selecting a reviewer from your group or team. However, you can also assign it to any reviewer. The list of reviewers can be found on [Engineering projects](https://about.gitlab.com/handbook/engineering/projects/) page. You can also use `workflow::ready for review` label. That means that your merge request is ready to be reviewed and any reviewer can pick it. It is recommended to use that label only if there isn't time pressure and make sure the merge request is assigned to a reviewer. diff --git a/doc/subscriptions/bronze_starter.md b/doc/subscriptions/bronze_starter.md index 78a9e324ada..2f33d7b740e 100644 --- a/doc/subscriptions/bronze_starter.md +++ b/doc/subscriptions/bronze_starter.md @@ -28,7 +28,7 @@ the tiers are no longer mentioned in GitLab documentation: - [Creating group memberships via CN](../user/group/index.md#create-group-links-via-cn) - [Group push rules](../user/group/index.md#group-push-rules) - [Managing group memberships via LDAP](../user/group/index.md#manage-group-memberships-via-ldap) - - [Member locking](../user/group/index.md#prevent-members-from-being-added-to-a-group) + - [Member locking](../user/group/index.md#prevent-members-from-being-added-to-projects-in-a-group) - [Overriding user permissions](../user/group/index.md#override-user-permissions) - [User contribution analytics](../user/group/contribution_analytics/index.md) - [Kerberos integration](../integration/kerberos.md) diff --git a/doc/user/group/index.md b/doc/user/group/index.md index b9539644b51..b665391622f 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -508,7 +508,7 @@ To prevent a project from being shared with other groups: This setting applies to all subgroups unless overridden by a group owner. Groups already added to a project lose access when the setting is enabled. -## Prevent members from being added to a group **(PREMIUM)** +## Prevent members from being added to projects in a group **(PREMIUM)** As a group owner, you can prevent any new project membership for all projects in a group, allowing tighter control over project membership. @@ -516,7 +516,11 @@ projects in a group, allowing tighter control over project membership. For example, if you want to lock the group for an [Audit Event](../../administration/audit_events.md), you can guarantee that project membership cannot be modified during the audit. -To prevent members from being added to a group: +You can still invite groups or to add members to groups, implicitly giving members access to projects in the **locked** group. + +The setting does not cascade. Projects in subgroups observe the subgroup configuration, ignoring the parent group. + +To prevent members from being added to projects in a group: 1. Go to the group's **Settings > General** page. 1. Expand the **Permissions, LFS, 2FA** section. diff --git a/locale/gitlab.pot b/locale/gitlab.pot index f0e3bcc8cb1..4cd6b1f2d9a 100644 --- a/locale/gitlab.pot +++ b/locale/gitlab.pot @@ -26247,7 +26247,7 @@ msgstr "" msgid "Prev" msgstr "" -msgid "Prevent adding new members to project membership within this group" +msgid "Prevent adding new members to projects within this group" msgstr "" msgid "Prevent auto-stopping" diff --git a/rubocop/cop/static_translation_definition.rb b/rubocop/cop/static_translation_definition.rb index ac50fd94884..3475a2b3dca 100644 --- a/rubocop/cop/static_translation_definition.rb +++ b/rubocop/cop/static_translation_definition.rb @@ -8,11 +8,15 @@ module RuboCop TRANSLATION_METHODS = %i[_ s_ n_].freeze def_node_matcher :translation_method?, <<~PATTERN - (send _ _ str*) + (send _ _ str*) PATTERN def_node_matcher :lambda_node?, <<~PATTERN - (send _ :lambda) + (send _ :lambda) + PATTERN + + def_node_matcher :struct_constant_assignment?, <<~PATTERN + (casgn _ _ `(const _ :Struct)) PATTERN def on_send(node) @@ -27,7 +31,7 @@ module RuboCop receiver, _ = *ancestor break if lambda_node?(receiver) # translations defined in lambda nodes should be allowed - if constant_assignment?(ancestor) + if constant_assignment?(ancestor) && !struct_constant_assignment?(ancestor) add_offense(node, location: :expression) break diff --git a/spec/fixtures/markdown/markdown_golden_master_examples.yml b/spec/fixtures/markdown/markdown_golden_master_examples.yml new file mode 100644 index 00000000000..d0ccf46f69c --- /dev/null +++ b/spec/fixtures/markdown/markdown_golden_master_examples.yml @@ -0,0 +1,792 @@ +# Related Specs: +# +# This data file drives the specs in the following specs: +# +# CE Backend: spec/requests/api/markdown_golden_master_spec.rb +# CE Frontend: spec/frontend/content_editor/markdown_processing_spec.js +# +# For EE, these files are used: +# EE Data: ee/spec/fixtures/markdown/markdown_golden_master_examples.yml +# EE Backend: ee/spec/requests/api/markdown_golden_master_spec.rb +# EE Frontend: ee/spec/frontend/content_editor/markdown_processing_spec.js +# +# +# Requirements: +# +# 1. Frontend: We should have test coverage that the Content Editor can properly serialize HTML +# to Markdown for all GFM source elements which it currently supports. +# 2. Frontend: We should have test coverage that the Content Editor can properly render the expected +# HTML for all GFM source elements which it currently supports (not currently implemented in the +# frontend - this will likely be a standalone module outside of the Content Editor). +# 3. Backend: We should ensure that for all GFM elements, the backend always renders the expected +# HTML, for **all** supported GFM source elements. +# +# If any of this this ever changes unexpectedly, tests will start failing, and force the same change +# to be made on the backend and frontend. +# +# +# Overview: +# +# These specs ensure that the bidirectional Markdown <-> HTML conversion logic is implemented +# identically on the backend and frontend, for all supported GitLab-Flavored Markdown examples, by +# running hardcoded examples through the logic and ensuring the results match. +# +# This is an example of the "Golden Master Testing" approach, which is also referred to as +# "Approval Testing" or "Characterization Testing". +# +# The term "Golden Master" originally comes from the recording industry, and refers to process +# of "mastering", or making a final mix from which all other copies will be produced. +# +# See: +# - https://en.wikipedia.org/wiki/Characterization_test +# - https://en.wikipedia.org/wiki/Gold_master_(disambiguation) +# +# +# What we are doing is actually a type Golden Master testing with modifications: +# +# 1. The original markdown examples used to drive the tests are taken from this YAML, and can be +# considered a form of "fixture" in this case. +# 2. The HTML in the YAML is the "Golden Master", but we are going to use it to assert +# against **TWO** different implementations of markdown rendering: +# 1. The frontend, implemented as Jest specs. +# 1. This will assert both HTML -> markdown serialization (what it currently does), as well as... +# 2. Markdown -> HTML rendering (not currently implemented in the frontend - this will likely +# be a standalone module outside of the Content Editor) +# 1. The backend, implemented as requests specs +# 1. This will assert markdown -> HTML conversion by the backend. +# +# Also see the MR for more explanation on the details of this approach: +# https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68671 +# +# +# Usage: +# +# * Please keep this file alphabetized. +# * To run focused example(s), set the `FOCUSED_MARKDOWN_EXAMPLES` environment variable to a +# comma-separated list of example names. This works for the frontend and backend specs. +# * Required attributes for every example: +# 1. `name`: Specifies the Name of the example, which will be printed when specs are run. +# 2. `markdown`: Specifies the Markdown for the example, which will be compared with the +# Markdown the code generates from the corresponding specified HTML. +# 3. `html`: Specifies the HTML for the example, which will be compared with the +# HTML the code generated from the corresponding specified Markdown. +# * `api_context` (optional): This is used when a single markdown can be +# rendered differently depending on the API endpoint or area of the app from which it is called or +# used. The valid values for `api_context` are: `project`, `group`, `project_wiki`, +# and (for EE only) `group_wiki`.The `name` attribute must also have a `_for_[API_CONTEXT]` suffix +# which matches the `api_context`, in order to ensure that each example has a unique `name` +# identifier. For example, `attachment_image_for_project`. +# * `pending`: To skip an example that is broken or not yet fully implemented, add +# a `pending: ` attribute to the example. See +# the `a_example_of_pending` entry for an example. +# * `pending` with key: You can also mark an example pending on only the frontend or backend. See +# the `a_example_of_pending_with_keys` entry for an example. +# * `substitutions`: For examples which may have variable content in different environments, +# such as portions of the URI, or database record IDs, you can specify +# `substitutions`, which is an array of regex/replacement pairs. The HTML +# value will be normalized with each of these pairs using Ruby `gsub` +# before comparing. +# The substitution values can (and are) also reused in multiple examples +# via YAML anchors. +# +# +# Notes: +# +# * The html values should exactly match what the backend markdown API endpoints return for the +# given markdown example. The HTML is intentionally not indented, formatted, or split across lines. +# This is a bit less readable, but it makes the spec logic simpler and less error prone for edge +# cases. +# +# +# Debugging Failures and Writing New Entries: +# +# * You need to compare what is different between the expected and actual values. +# * In rspec, the diff printed out includes the full text of the HTML. This may be long, so you +# may want to turn line wrapping on or off or copy the diff to separate file(s) for easier comparison. +# * If the difference is just in an attribute value, use the `substitutions` support to normalize +# the HTML before comparing. These specs are only validating the HTML structure, the individual +# markdown elements' unit tests can provide coverage that the exact attribute values are correct. +# * If you are making a new entry, you can create the entry according to the `Usage` section above, +# but leave the `html` value blank. This will cause the spec to fail, and you can fill in the +# `html` value based on the spec failure that is printed out. + +--- + +#- name: an_example_of_pending +# pending: 'This is an example of the pending attribute: http://example.com' +# markdown: ;) +# html: |- +# +# + +#- name: an_example_of_pending_with_keys +# pending: +# frontend: 'This is an example of the frontend-only pending attribute: http://example.com' +# backend: 'This is an example of the backend-only pending attribute: http://example.com' +# markdown: ;) +# html: |- +# + +- name: attachment_image_for_group + api_context: group + substitutions: + # Note: having the top level `substitutions` data structure be a hash of arrays + # allows us to compose multiple substitutions via YAML anchors (YAML anchors + # pointing to arrays can't be combined) + uri_substitution: &uri_substitution + # NOTE: We don't care about verifying specific attribute values here, that should be the + # responsibility of unit tests. These tests are about the structure of the HTML. + - regex: '(href|data-src)(=")(.*?)(test-file\.(png|zip)")' + replacement: '\1\2URI_PREFIX\4' + markdown: |- + ![test-file](/uploads/aa45a38ec2cfe97433281b10bbff042c/test-file.png) + html: |- +

test-file

+ +- name: attachment_image_for_project + api_context: project + substitutions: + uri_substitution: *uri_substitution + markdown: |- + ![test-file](/uploads/aa45a38ec2cfe97433281b10bbff042c/test-file.png) + html: |- +

test-file

+ +- name: attachment_image_for_project_wiki + api_context: project_wiki + substitutions: + uri_substitution: *uri_substitution + markdown: |- + ![test-file](test-file.png) + html: |- +

test-file

+ +- name: attachment_link_for_group + api_context: group + substitutions: + uri_substitution: *uri_substitution + markdown: |- + [test-file](/uploads/aa45a38ec2cfe97433281b10bbff042c/test-file.zip) + html: |- +

test-file

+ +- name: attachment_link_for_project + api_context: project + substitutions: + uri_substitution: *uri_substitution + markdown: |- + [test-file](/uploads/aa45a38ec2cfe97433281b10bbff042c/test-file.zip) + html: |- +

test-file

+ +- name: attachment_link_for_project_wiki + api_context: project_wiki + substitutions: + uri_substitution: *uri_substitution + # TODO: The current frontend example doesn't include the path, need to look into why it does after refactoring to the new golden master approach + pending: + frontend: 'The current frontend example doesnt include the path, need to look into why it does after refactoring to the new golden master approach' + markdown: |- + [test-file](test-file.zip) + html: |- +

test-file

+ +- name: audio + markdown: |- + ![Sample Audio](https://gitlab.com/gitlab.mp3) + html: |- +

Sample Audio

+ +- name: audio_and_video_in_lists + markdown: |- + * ![Sample Audio](https://gitlab.com/1.mp3) + * ![Sample Video](https://gitlab.com/2.mp4) + + 1. ![Sample Video](https://gitlab.com/1.mp4) + 2. ![Sample Audio](https://gitlab.com/2.mp3) + + * [x] ![Sample Audio](https://gitlab.com/1.mp3) + * [x] ![Sample Audio](https://gitlab.com/2.mp3) + * [x] ![Sample Video](https://gitlab.com/3.mp4) + html: |- + +
    +
  1. Sample Video
    2. +
  2. Sample Audio
    3. +
+ + +- name: blockquote + markdown: |- + > This is a blockquote + > + > This is another one + html: |- +
+

This is a blockquote

+

This is another one

+
+ +- name: bold + markdown: |- + **bold** + html: |- +

bold

+ +- name: bullet_list_style_1 + markdown: |- + * list item 1 + * list item 2 + * embedded list item 3 + html: |- + + +- name: bullet_list_style_2 + markdown: |- + - list item 1 + - list item 2 + * embedded list item 3 + html: |- + + +- name: bullet_list_style_3 + markdown: |- + + list item 1 + + list item 2 + - embedded list item 3 + html: |- + + +- name: code_block + markdown: |- + ```javascript + console.log('hello world') + ``` + html: |- + 
  console.log('hello world')
+ +- name: color_chips + markdown: |- + - `#F00` + - `#F00A` + - `#FF0000` + - `#FF0000AA` + - `RGB(0,255,0)` + - `RGB(0%,100%,0%)` + - `RGBA(0,255,0,0.3)` + - `HSL(540,70%,50%)` + - `HSLA(540,70%,50%,0.3)` + html: |- + + +- name: description_list + markdown: |- +
+
Frog
+
Wet green thing
+
Rabbit
+
Warm fluffy thing
+
Punt
+
Kick a ball
+
Take a bet
+
Color
+
Colour
+
+ + Any hue except _white_ or **black** + +
+
+ html: |- +
+
Frog
+
Wet green thing
+
Rabbit
+
Warm fluffy thing
+
Punt
+
Kick a ball
+
Take a bet
+
Color
+
Colour
+
+

Any hue except white or black

+
+
+ +- name: details + markdown: |- +
+ This is the visible summary of the collapsible section + + 1. collapsed markdown + 2. more collapsed markdown + +
+ html: |- +
+ This is the visible summary of the collapsible section +
    +
  1. collapsed markdown
    2. +
  2. more collapsed markdown
    3. +
+
+ +- name: div + markdown: |- +
plain text
+
+ + just a plain ol' div, not much to _expect_! + +
+ html: |- +
plain text
+
+

just a plain ol' div, not much to expect!

+
+ +- name: emoji + markdown: |- + :sparkles: :heart: :100: + html: |- +

โœจ โค ๐Ÿ’ฏ

+ +- name: emphasis + markdown: _emphasized text_ + html:

emphasized text

+ +- name: figure + markdown: |- +
+ + ![Elephant at sunset](elephant-sunset.jpg) + +
An elephant at sunset
+
+
+ + ![A crocodile wearing crocs](croc-crocs.jpg) + +
+ + A crocodile wearing _crocs_! + +
+
+ html: |- +
+

Elephant at sunset

+
An elephant at sunset
+
+
+

A crocodile wearing crocs

+
+

A crocodile wearing crocs!

+
+
+ +- name: frontmatter_json + markdown: |- + ;;; + { + "title": "Page title" + } + ;;; + html: |- + 
{
+      "title": "Page title"
+    }
+ +- name: frontmatter_toml + markdown: |- + +++ + title = "Page title" + +++ + html: 
title = "Page title"
+ +- name: frontmatter_yaml + markdown: |- + --- + title: Page title + --- + html: 
title: Page title
+ +- name: hard_break + markdown: |- + This is a line after a\ + hard break + html: |- +

This is a line after a
+ hard break

+ +- name: headings + markdown: |- + # Heading 1 + + ## Heading 2 + + ### Heading 3 + + #### Heading 4 + + ##### Heading 5 + + ###### Heading 6 + html: |- +

+ Heading 1

+

+ Heading 2

+

+ Heading 3

+

+ Heading 4

+
+ Heading 5
+
+ Heading 6
+ +- name: horizontal_rule + markdown: |- + --- + html: |- +
+ +- name: html_marks + markdown: |- + * Content editor is ~~great~~amazing. + * If the changes LGTM, please MWPS. + * The English song Oh I do like to be beside the seaside looks like this in Hebrew: ืื”, ืื ื™ ืื•ื”ื‘ ืœื”ื™ื•ืช ืœื™ื“ ื—ื•ืฃ ื”ื™ื. In the computer's memory, this is stored as ืื”, ืื ื™ ืื•ื”ื‘ ืœื”ื™ื•ืช ืœื™ื“ ื—ื•ืฃ ื”ื™ื. + * The Scream by Edvard Munch. Painted in 1893. + * HTML is the standard markup language for creating web pages. + * Do not forget to buy milk today. + * This is a paragraph and smaller text goes here. + * The concert starts at and you'll be able to enjoy the band for at least . + * Press Ctrl + C to copy text (Windows). + * WWF's goal is to: Build a future where people live in harmony with nature. We hope they succeed. + * The error occured was: Keyboard not found. Press F1 to continue. + * The area of a triangle is: 1/2 x b x h, where b is the base, and h is the vertical height. + * ๆผขใ„ใ„ขห‹ + * C7H16 + O2 โ†’ CO2 + H2O + * The **Pythagorean theorem** is often expressed as a2 + b2 = c2 + html: |- + + +- name: image + markdown: |- + ![alt text](https://gitlab.com/logo.png) + html: |- +

alt text

+ +- name: inline_code + markdown: |- + `code` + html: |- +

code

+ +- name: inline_diff + markdown: |- + * {-deleted-} + * {+added+} + html: |- + + +- name: label + pending: + # TODO: There is an error with the frontend HTML to markdown spec adding a double escape (\\) to the label tilde. + frontend: 'There is an error with the frontend HTML to markdown spec adding a double escape (\\) to the label tilde.' + markdown: |- + ~bug + html: |- +

~bug

+ +- name: link + markdown: |- + [GitLab](https://gitlab.com) + html: |- +

GitLab

+ +- name: math + markdown: |- + This math is inline $`a^2+b^2=c^2`$. + + This is on a separate line: + + ```math + a^2+b^2=c^2 + ``` + html: |- +

This math is inline a^2+b^2=c^2.

+

This is on a separate line:

+ 
a^2+b^2=c^2
+ +- name: ordered_list + markdown: |- + 1. list item 1 + 2. list item 2 + 3. list item 3 + html: |- +
    +
  1. list item 1
    2. +
  2. list item 2
    3. +
  3. list item 3
    4. +
+ +- name: ordered_list_with_start_order + markdown: |- + 134. list item 1 + 135. list item 2 + 136. list item 3 + html: |- +
    +
  1. list item 1
    2. +
  2. list item 2
    3. +
  3. list item 3
    4. +
+ +- name: ordered_task_list + markdown: |- + 1. [x] hello + 2. [x] world + 3. [ ] example + 1. [ ] of nested + 1. [x] task list + 2. [ ] items + html: |- +
    +
  1. + hello
    2. +
  2. + world
    3. +
  3. + example +
      +
    1. + of nested +
        +
      1. + task list
        2. +
      2. + items
        3. +
      +
      2. +
    +
    4. +
+ +- name: ordered_task_list_with_order + markdown: |- + 4893. [x] hello + 4894. [x] world + 4895. [ ] example + html: |- +
    +
  1. + hello
    2. +
  2. + world
    3. +
  3. + example
    4. +
+ +- name: reference_for_project_wiki + api_context: project_wiki + substitutions: + # NOTE: We don't care about verifying specific attribute values here, that should be the + # responsibility of unit tests. These tests are about the structure of the HTML. + uri_substitution: *uri_substitution + data_attribute_id_substitution: + - regex: '(data-user|data-project|data-issue|data-iid|data-merge-request|data-milestone)(=")(\d+?)(")' + replacement: '\1\2ID\4' + text_attribute_substitution: + - regex: '(title)(=")(.+?)(")' + replacement: '\1\2TEXT\4' + path_attribute_id_substitution: + - regex: '(group|project)(\d+)' + replacement: '\1ID' + markdown: |- + Hi @gfm_user - thank you for reporting this bug (#1) we hope to fix it in %1.1 as part of !1 + html: |- +

Hi @gfm_user - thank you for reporting this bug (#1) we hope to fix it in %1.1 as part of !1

+- name: strike + markdown: |- + ~~del~~ + html: |- +

del

+ +- name: table + markdown: |- + | header | header | + |--------|--------| + | `code` | cell with **bold** | + | ~~strike~~ | cell with _italic_ | + + # content after table + html: |- + + + + + + + + + + + + + + + + + +
headerheader
codecell with bold +
strikecell with italic +
+

+ content after table

+ +- name: table_of_contents + markdown: |- + [[_TOC_]] + + # Lorem + + Well, that's just like... your opinion.. man. + + ## Ipsum + + ### Dolar + + # Sit amit + + ### I don't know + html: |- + +

+ Lorem

+

Well, that's just like... your opinion.. man.

+

+ Ipsum

+

+ Dolar

+

+ Sit amit

+

+ I don't know

+ +- name: task_list + markdown: |- + * [x] hello + * [x] world + * [ ] example + * [ ] of nested + * [x] task list + * [ ] items + html: |- + + +- name: video + markdown: |- + ![Sample Video](https://gitlab.com/gitlab.mp4) + html: |- +

Sample Video

+ +- name: word_break + markdown: FernstraรŸenbauprivatfinanzierungsgesetz + html:

FernstraรŸenbauprivatfinanzierungsgesetz

diff --git a/spec/frontend/content_editor/markdown_processing_examples.js b/spec/frontend/content_editor/markdown_processing_examples.js deleted file mode 100644 index da895970289..00000000000 --- a/spec/frontend/content_editor/markdown_processing_examples.js +++ /dev/null @@ -1,27 +0,0 @@ -import fs from 'fs'; -import path from 'path'; -import jsYaml from 'js-yaml'; -// eslint-disable-next-line import/no-deprecated -import { getJSONFixture } from 'helpers/fixtures'; - -export const loadMarkdownApiResult = (testName) => { - const fixturePathPrefix = `api/markdown/${testName}.json`; - - // eslint-disable-next-line import/no-deprecated - const fixture = getJSONFixture(fixturePathPrefix); - return fixture.body || fixture.html; -}; - -export const loadMarkdownApiExamples = () => { - const apiMarkdownYamlPath = path.join(__dirname, '..', 'fixtures', 'api_markdown.yml'); - const apiMarkdownYamlText = fs.readFileSync(apiMarkdownYamlPath); - const apiMarkdownExampleObjects = jsYaml.safeLoad(apiMarkdownYamlText); - - return apiMarkdownExampleObjects.map(({ name, context, markdown }) => [name, context, markdown]); -}; - -export const loadMarkdownApiExample = (testName) => { - return loadMarkdownApiExamples().find(([name, context]) => { - return (context ? `${context}_${name}` : name) === testName; - })[2]; -}; diff --git a/spec/frontend/content_editor/markdown_processing_spec.js b/spec/frontend/content_editor/markdown_processing_spec.js index 71565768558..4bbe5b4bcf4 100644 --- a/spec/frontend/content_editor/markdown_processing_spec.js +++ b/spec/frontend/content_editor/markdown_processing_spec.js @@ -1,20 +1,19 @@ -import { createContentEditor } from '~/content_editor'; -import { loadMarkdownApiExamples, loadMarkdownApiResult } from './markdown_processing_examples'; +import path from 'path'; +import { createSharedExamples, loadMarkdownApiExamples } from './markdown_processing_spec_helper'; jest.mock('~/emoji'); -describe('markdown processing', () => { +// See spec/fixtures/markdown/markdown_golden_master_examples.yml for documentation on how this spec works. +describe('markdown processing in ContentEditor', () => { // Ensure we generate same markdown that was provided to Markdown API. - it.each(loadMarkdownApiExamples())( - 'correctly handles %s (context: %s)', - async (name, context, markdown) => { - const testName = context ? `${context}_${name}` : name; - const contentEditor = createContentEditor({ - renderMarkdown: () => loadMarkdownApiResult(testName), - }); - await contentEditor.setSerializedContent(markdown); - - expect(contentEditor.getSerializedContent()).toBe(markdown); - }, + const markdownYamlPath = path.join( + __dirname, + '..', + '..', + 'fixtures', + 'markdown', + 'markdown_golden_master_examples.yml', ); + // eslint-disable-next-line jest/valid-describe + describe.each(loadMarkdownApiExamples(markdownYamlPath))('%s', createSharedExamples); }); diff --git a/spec/frontend/content_editor/markdown_processing_spec_helper.js b/spec/frontend/content_editor/markdown_processing_spec_helper.js new file mode 100644 index 00000000000..66eccbe8a4e --- /dev/null +++ b/spec/frontend/content_editor/markdown_processing_spec_helper.js @@ -0,0 +1,75 @@ +import fs from 'fs'; +import jsYaml from 'js-yaml'; +import { memoize } from 'lodash'; +import { createContentEditor } from '~/content_editor'; +import { setTestTimeoutOnce } from 'helpers/timeout'; + +const getFocusedMarkdownExamples = memoize( + () => process.env.FOCUSED_MARKDOWN_EXAMPLES?.split(',') || [], +); + +const includeExample = ({ name }) => { + const focusedMarkdownExamples = getFocusedMarkdownExamples(); + if (!focusedMarkdownExamples.length) { + return true; + } + return focusedMarkdownExamples.includes(name); +}; + +const getPendingReason = (pendingStringOrObject) => { + if (!pendingStringOrObject) { + return null; + } + if (typeof pendingStringOrObject === 'string') { + return pendingStringOrObject; + } + if (pendingStringOrObject.frontend) { + return pendingStringOrObject.frontend; + } + + return null; +}; + +// eslint-disable-next-line jest/no-export +export const loadMarkdownApiExamples = (markdownYamlPath) => { + const apiMarkdownYamlText = fs.readFileSync(markdownYamlPath); + const apiMarkdownExampleObjects = jsYaml.safeLoad(apiMarkdownYamlText); + + return apiMarkdownExampleObjects + .filter(includeExample) + .map(({ name, pending, markdown, html }) => [ + name, + { pendingReason: getPendingReason(pending), markdown, html }, + ]); +}; + +const testSerializesHtmlToMarkdownForElement = async ({ markdown, html }) => { + const contentEditor = createContentEditor({ + // Overwrite renderMarkdown to always return this specific html + renderMarkdown: () => html, + }); + + await contentEditor.setSerializedContent(markdown); + + // This serializes the ContentEditor document, which was based on the HTML, to markdown + const serializedContent = contentEditor.getSerializedContent(); + + // Assert that the markdown we ended up with after sending it through all the ContentEditor + // plumbing matches the original markdown from the YAML. + expect(serializedContent).toBe(markdown); +}; + +// eslint-disable-next-line jest/no-export +export const createSharedExamples = (name, { pendingReason, ...example }) => { + const exampleName = 'correctly serializes HTML to markdown'; + if (pendingReason) { + it.todo(`${exampleName}: ${pendingReason}`); + } else { + it(exampleName, async () => { + if (name === 'frontmatter_toml') { + setTestTimeoutOnce(2000); + } + await testSerializesHtmlToMarkdownForElement(example); + }); + } +}; diff --git a/spec/frontend/fixtures/api_markdown.rb b/spec/frontend/fixtures/api_markdown.rb deleted file mode 100644 index 89f012a5110..00000000000 --- a/spec/frontend/fixtures/api_markdown.rb +++ /dev/null @@ -1,65 +0,0 @@ -# frozen_string_literal: true - -require 'spec_helper' - -RSpec.describe API::MergeRequests, '(JavaScript fixtures)', type: :request do - include ApiHelpers - include WikiHelpers - include JavaScriptFixturesHelpers - - let_it_be(:user) { create(:user, username: 'gitlab') } - - let_it_be(:group) { create(:group, :public) } - let_it_be(:project) { create(:project, :public, :repository, group: group) } - - let_it_be(:label) { create(:label, project: project, title: 'bug') } - let_it_be(:milestone) { create(:milestone, project: project, title: '1.1') } - let_it_be(:issue) { create(:issue, project: project) } - let_it_be(:merge_request) { create(:merge_request, source_project: project) } - - let_it_be(:project_wiki) { create(:project_wiki, project: project, user: user) } - - let(:project_wiki_page) { create(:wiki_page, wiki: project_wiki) } - - before(:all) do - group.add_owner(user) - project.add_maintainer(user) - end - - before do - sign_in(user) - end - - markdown_examples = begin - yaml_file_path = File.expand_path('api_markdown.yml', __dir__) - yaml = File.read(yaml_file_path) - YAML.safe_load(yaml, symbolize_names: true) - end - - markdown_examples.each do |markdown_example| - context = markdown_example.fetch(:context, '') - name = markdown_example.fetch(:name) - - context "for #{name}#{!context.empty? ? " (context: #{context})" : ''}" do - let(:markdown) { markdown_example.fetch(:markdown) } - - name = "#{context}_#{name}" unless context.empty? - - it "api/markdown/#{name}.json" do - api_url = case context - when 'project' - "/#{project.full_path}/preview_markdown" - when 'group' - "/groups/#{group.full_path}/preview_markdown" - when 'project_wiki' - "/#{project.full_path}/-/wikis/#{project_wiki_page.slug}/preview_markdown" - else - api "/markdown" - end - - post api_url, params: { text: markdown, gfm: true } - expect(response).to be_successful - end - end - end -end diff --git a/spec/frontend/fixtures/api_markdown.yml b/spec/frontend/fixtures/api_markdown.yml deleted file mode 100644 index 8fd6a5531db..00000000000 --- a/spec/frontend/fixtures/api_markdown.yml +++ /dev/null @@ -1,289 +0,0 @@ -# This data file drives the specs in -# spec/frontend/fixtures/api_markdown.rb and -# spec/frontend/content_editor/extensions/markdown_processing_spec.js ---- -- name: attachment_image - context: group - markdown: '![test-file](/uploads/aa45a38ec2cfe97433281b10bbff042c/test-file.png)' -- name: attachment_image - context: project - markdown: '![test-file](/uploads/aa45a38ec2cfe97433281b10bbff042c/test-file.png)' -- name: attachment_image - context: project_wiki - markdown: '![test-file](test-file.png)' -- name: attachment_link - context: group - markdown: '[test-file](/uploads/aa45a38ec2cfe97433281b10bbff042c/test-file.zip)' -- name: attachment_link - context: project - markdown: '[test-file](/uploads/aa45a38ec2cfe97433281b10bbff042c/test-file.zip)' -- name: attachment_link - context: project_wiki - markdown: '[test-file](test-file.zip)' -- name: audio - markdown: '![Sample Audio](https://gitlab.com/gitlab.mp3)' -- name: audio_and_video_in_lists - markdown: |- - * ![Sample Audio](https://gitlab.com/1.mp3) - * ![Sample Video](https://gitlab.com/2.mp4) - - 1. ![Sample Video](https://gitlab.com/1.mp4) - 2. ![Sample Audio](https://gitlab.com/2.mp3) - - * [x] ![Sample Audio](https://gitlab.com/1.mp3) - * [x] ![Sample Audio](https://gitlab.com/2.mp3) - * [x] ![Sample Video](https://gitlab.com/3.mp4) -- name: blockquote - markdown: |- - > This is a blockquote - > - > This is another one -- name: bold - markdown: '**bold**' -- name: bullet_list_style_1 - markdown: |- - * list item 1 - * list item 2 - * embedded list item 3 -- name: bullet_list_style_2 - markdown: |- - - list item 1 - - list item 2 - * embedded list item 3 -- name: bullet_list_style_3 - markdown: |- - + list item 1 - + list item 2 - - embedded list item 3 -- name: code_block - markdown: |- - ```javascript - console.log('hello world') - ``` -- name: color_chips - markdown: |- - - `#F00` - - `#F00A` - - `#FF0000` - - `#FF0000AA` - - `RGB(0,255,0)` - - `RGB(0%,100%,0%)` - - `RGBA(0,255,0,0.3)` - - `HSL(540,70%,50%)` - - `HSLA(540,70%,50%,0.3)` -- name: description_list - markdown: |- -
-
Frog
-
Wet green thing
-
Rabbit
-
Warm fluffy thing
-
Punt
-
Kick a ball
-
Take a bet
-
Color
-
Colour
-
- - Any hue except _white_ or **black** - -
-
-- name: details - markdown: |- -
- Apply this patch - - ```diff - diff --git a/spec/frontend/fixtures/api_markdown.yml b/spec/frontend/fixtures/api_markdown.yml - index 8433efaf00c..69b12c59d46 100644 - --- a/spec/frontend/fixtures/api_markdown.yml - +++ b/spec/frontend/fixtures/api_markdown.yml - @@ -33,6 +33,13 @@ - * ๆผขใ„ใ„ขห‹ - * C7H16 + O2 โ†’ CO2 + H2O - * The **Pythagorean theorem** is often expressed as a2 + b2 = c2.The **Pythagorean theorem** is often expressed as a2 + b2 = c2 - +- name: details - + markdown: |- - +
- + Apply this patch - + - + ๐Ÿถ much meta, ๐Ÿถ many patch - + ๐Ÿถ such diff, ๐Ÿถ very meme - + ๐Ÿถ wow! - +
- - name: link - markdown: '[GitLab](https://gitlab.com)' - - name: attachment_link - ``` - -
-- name: div - markdown: |- -
plain text
-
- - just a plain ol' div, not much to _expect_! - -
-- name: emoji - markdown: ':sparkles: :heart: :100:' -- name: emphasis - markdown: '_emphasized text_' -- name: figure - markdown: |- -
- - ![Elephant at sunset](elephant-sunset.jpg) - -
An elephant at sunset
-
-
- - ![A crocodile wearing crocs](croc-crocs.jpg) - -
- - A crocodile wearing _crocs_! - -
-
-- name: frontmatter_json - markdown: |- - ;;; - { - "title": "Page title" - } - ;;; -- name: frontmatter_toml - markdown: |- - +++ - title = "Page title" - +++ -- name: frontmatter_yaml - markdown: |- - --- - title: Page title - --- -- name: hard_break - markdown: |- - This is a line after a\ - hard break -- name: headings - markdown: |- - # Heading 1 - - ## Heading 2 - - ### Heading 3 - - #### Heading 4 - - ##### Heading 5 - - ###### Heading 6 -- name: horizontal_rule - markdown: '---' -- name: html_marks - markdown: |- - * Content editor is ~~great~~amazing. - * If the changes LGTM, please MWPS. - * The English song Oh I do like to be beside the seaside looks like this in Hebrew: ืื”, ืื ื™ ืื•ื”ื‘ ืœื”ื™ื•ืช ืœื™ื“ ื—ื•ืฃ ื”ื™ื. In the computer's memory, this is stored as ืื”, ืื ื™ ืื•ื”ื‘ ืœื”ื™ื•ืช ืœื™ื“ ื—ื•ืฃ ื”ื™ื. - * The Scream by Edvard Munch. Painted in 1893. - * HTML is the standard markup language for creating web pages. - * Do not forget to buy milk today. - * This is a paragraph and smaller text goes here. - * The concert starts at and you'll be able to enjoy the band for at least . - * Press Ctrl + C to copy text (Windows). - * WWF's goal is to: Build a future where people live in harmony with nature. We hope they succeed. - * The error occured was: Keyboard not found. Press F1 to continue. - * The area of a triangle is: 1/2 x b x h, where b is the base, and h is the vertical height. - * ๆผขใ„ใ„ขห‹ - * C7H16 + O2 โ†’ CO2 + H2O - * The **Pythagorean theorem** is often expressed as a2 + b2 = c2 -- name: image - markdown: '![alt text](https://gitlab.com/logo.png)' -- name: inline_code - markdown: '`code`' -- name: inline_diff - markdown: |- - * {-deleted-} - * {+added+} -- name: link - markdown: '[GitLab](https://gitlab.com)' -- name: math - markdown: |- - This math is inline $`a^2+b^2=c^2`$. - - This is on a separate line: - - ```math - a^2+b^2=c^2 - ``` -- name: ordered_list - markdown: |- - 1. list item 1 - 2. list item 2 - 3. list item 3 -- name: ordered_list_with_start_order - markdown: |- - 134. list item 1 - 135. list item 2 - 136. list item 3 -- name: ordered_task_list - markdown: |- - 1. [x] hello - 2. [x] world - 3. [ ] example - 1. [ ] of nested - 1. [x] task list - 2. [ ] items -- name: ordered_task_list_with_order - markdown: |- - 4893. [x] hello - 4894. [x] world - 4895. [ ] example -- name: reference - context: project_wiki - markdown: |- - Hi @gitlab - thank you for reporting this ~bug (#1) we hope to fix it in %1.1 as part of !1 -- name: strike - markdown: '~~del~~' -- name: table - markdown: |- - | header | header | - |--------|--------| - | `code` | cell with **bold** | - | ~~strike~~ | cell with _italic_ | - - # content after table -- name: table_of_contents - markdown: |- - [[_TOC_]] - - # Lorem - - Well, that's just like... your opinion.. man. - - ## Ipsum - - ### Dolar - - # Sit amit - - ### I don't know -- name: task_list - markdown: |- - * [x] hello - * [x] world - * [ ] example - * [ ] of nested - * [x] task list - * [ ] items -- name: thematic_break - markdown: |- - --- -- name: video - markdown: '![Sample Video](https://gitlab.com/gitlab.mp4)' -- name: word_break - markdown: FernstraรŸenbauprivatfinanzierungsgesetz diff --git a/spec/requests/api/markdown_golden_master_spec.rb b/spec/requests/api/markdown_golden_master_spec.rb new file mode 100644 index 00000000000..4fa946de342 --- /dev/null +++ b/spec/requests/api/markdown_golden_master_spec.rb @@ -0,0 +1,9 @@ +# frozen_string_literal: true + +require 'spec_helper' + +# See spec/fixtures/markdown/markdown_golden_master_examples.yml for documentation on how this spec works. +RSpec.describe API::Markdown, 'Golden Master' do + markdown_yml_file_path = File.expand_path('../../fixtures/markdown/markdown_golden_master_examples.yml', __dir__) + include_context 'API::Markdown Golden Master shared context', markdown_yml_file_path +end diff --git a/spec/rubocop/cop/static_translation_definition_spec.rb b/spec/rubocop/cop/static_translation_definition_spec.rb index b2b04cbcbde..554a7c17a4b 100644 --- a/spec/rubocop/cop/static_translation_definition_spec.rb +++ b/spec/rubocop/cop/static_translation_definition_spec.rb @@ -112,7 +112,7 @@ RSpec.describe RuboCop::Cop::StaticTranslationDefinition do } end CODE - <<~CODE + <<~CODE, class MyClass def hello { @@ -121,6 +121,20 @@ RSpec.describe RuboCop::Cop::StaticTranslationDefinition do end end CODE + <<~CODE, + SomeClass = Struct.new do + def text + _('Some translated text') + end + end + CODE + <<~CODE + Struct.new('SomeClass') do + def text + _('Some translated text') + end + end + CODE ] end diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb index 922a615cc3f..5e85387ea14 100644 --- a/spec/spec_helper.rb +++ b/spec/spec_helper.rb @@ -450,6 +450,13 @@ RSpec.configure do |config| $stdout = StringIO.new end + # Makes diffs show entire non-truncated values. + config.before(:each, unlimited_max_formatted_output_length: true) do |_example| + config.expect_with :rspec do |c| + c.max_formatted_output_length = nil + end + end + config.after(:each, :silence_stdout) do $stdout = STDOUT end diff --git a/spec/support/shared_contexts/markdown_golden_master_shared_examples.rb b/spec/support/shared_contexts/markdown_golden_master_shared_examples.rb new file mode 100644 index 00000000000..d0915bbf158 --- /dev/null +++ b/spec/support/shared_contexts/markdown_golden_master_shared_examples.rb @@ -0,0 +1,127 @@ +# frozen_string_literal: true + +require 'spec_helper' + +# See spec/fixtures/markdown/markdown_golden_master_examples.yml for documentation on how this spec works. +RSpec.shared_context 'API::Markdown Golden Master shared context' do |markdown_yml_file_path| + include ApiHelpers + include WikiHelpers + + let_it_be(:user) { create(:user, username: 'gfm_user') } + + let_it_be(:group) { create(:group, :public) } + let_it_be(:project) { create(:project, :public, :repository, group: group) } + + let_it_be(:label) { create(:label, project: project, title: 'bug') } + let_it_be(:milestone) { create(:milestone, project: project, title: '1.1') } + let_it_be(:issue) { create(:issue, project: project) } + let_it_be(:merge_request) { create(:merge_request, source_project: project) } + + let_it_be(:project_wiki) { create(:project_wiki, project: project, user: user) } + + let_it_be(:project_wiki_page) { create(:wiki_page, wiki: project_wiki) } + + before(:all) do + group.add_owner(user) + project.add_maintainer(user) + end + + before do + sign_in(user) + end + + markdown_examples = begin + yaml = File.read(markdown_yml_file_path) + YAML.safe_load(yaml, symbolize_names: true, aliases: true) + end + + it "examples must be unique and alphabetized by name", :unlimited_max_formatted_output_length do + names = markdown_examples.map { |example| example[:name] } + expect(names).to eq(names.sort.uniq) + end + + if focused_markdown_examples_string = ENV['FOCUSED_MARKDOWN_EXAMPLES'] + focused_markdown_examples = focused_markdown_examples_string.split(',').map(&:strip) || [] + markdown_examples.reject! {|markdown_example| !focused_markdown_examples.include?(markdown_example.fetch(:name)) } + end + + markdown_examples.each do |markdown_example| + name = markdown_example.fetch(:name) + api_context = markdown_example[:api_context] + + if api_context && !name.end_with?("_for_#{api_context}") + raise "Name must have suffix of '_for_#{api_context}' to the api_context" + end + + context "for #{name}#{api_context ? " (api_context: #{api_context})" : ''}" do + let(:pending_reason) do + pending_value = markdown_example.fetch(:pending, nil) + get_pending_reason(pending_value) + end + + let(:example_markdown) { markdown_example.fetch(:markdown) } + let(:example_html) { markdown_example.fetch(:html) } + let(:substitutions) { markdown_example.fetch(:substitutions, {}) } + + it "verifies conversion of GFM to HTML", :unlimited_max_formatted_output_length do + pending pending_reason if pending_reason + + normalized_example_html = normalize_html(example_html, substitutions) + + api_url = get_url_for_api_context(api_context) + + post api_url, params: { text: example_markdown, gfm: true } + expect(response).to be_successful + response_body = Gitlab::Json.parse(response.body) + # Some requests have the HTML in the `html` key, others in the `body` key. + response_html = response_body['body'] ? response_body.fetch('body') : response_body.fetch('html') + normalized_response_html = normalize_html(response_html, substitutions) + + expect(normalized_response_html).to eq(normalized_example_html) + end + + def get_pending_reason(pending_value) + return false unless pending_value + + return pending_value if pending_value.is_a?(String) + + pending_value[:backend] || false + end + + def normalize_html(html, substitutions) + normalized_html = html.dup + # Note: having the top level `substitutions` data structure be a hash of arrays + # allows us to compose multiple substitutions via YAML anchors (YAML anchors + # pointing to arrays can't be combined) + substitutions.each_value do |substitution_entry| + substitution_entry.each do |substitution| + regex = substitution.fetch(:regex) + replacement = substitution.fetch(:replacement) + normalized_html.gsub!(%r{#{regex}}, replacement) + end + end + + normalized_html + end + end + end + + def supported_api_contexts + %w(project group project_wiki) + end + + def get_url_for_api_context(api_context) + case api_context + when 'project' + "/#{project.full_path}/preview_markdown" + when 'group' + "/groups/#{group.full_path}/preview_markdown" + when 'project_wiki' + "/#{project.full_path}/-/wikis/#{project_wiki_page.slug}/preview_markdown" + when nil + api "/markdown" + else + raise "Error: 'context' extension was '#{api_context}'. It must be one of: #{supported_api_contexts.join(',')}" + end + end +end