diff --git a/doc/development/new_fe_guide/development/testing.md b/doc/development/new_fe_guide/development/testing.md index f7ea496d935..e0d413b748b 100644 --- a/doc/development/new_fe_guide/development/testing.md +++ b/doc/development/new_fe_guide/development/testing.md @@ -1,361 +1,6 @@ -# Overview of Frontend Testing +--- +redirect_to: '../../testing_guide/frontend_testing.md' +--- -Tests relevant for frontend development can be found at the following places: +This document was moved to [another location](../../testing_guide/frontend_testing.md). -- `spec/javascripts/` which are run by Karma (command: `yarn karma`) and contain - - [frontend unit tests](#frontend-unit-tests) - - [frontend component tests](#frontend-component-tests) - - [frontend integration tests](#frontend-integration-tests) -- `spec/frontend/` which are run by Jest (command: `yarn jest`) and contain - - [frontend unit tests](#frontend-unit-tests) - - [frontend component tests](#frontend-component-tests) - - [frontend integration tests](#frontend-integration-tests) -- `spec/features/` which are run by RSpec and contain - - [feature tests](#feature-tests) - -All tests in `spec/javascripts/` will eventually be migrated to `spec/frontend/` (see also [#52483](https://gitlab.com/gitlab-org/gitlab-ce/issues/52483)). - -In addition there were feature tests in `features/` run by Spinach in the past. -These have been removed from our codebase in May 2018 ([#23036](https://gitlab.com/gitlab-org/gitlab-ce/issues/23036)). - -See also: - -- [Old testing guide](../../testing_guide/frontend_testing.html). -- [Notes on testing Vue components](../../fe_guide/vue.html#testing-vue-components). - -## Frontend unit tests - -Unit tests are on the lowest abstraction level and typically test functionality that is not directly perceivable by a user. - -### When to use unit tests - -
- exported functions and classes - Anything that is exported can be reused at various places in a way you have no control over. - Therefore it is necessary to document the expected behavior of the public interface with tests. -
- -
- Vuex actions - Any Vuex action needs to work in a consistent way independent of the component it is triggered from. -
- -
- Vuex mutations - For complex Vuex mutations it helps to identify the source of a problem by separating the tests from other parts of the Vuex store. -
- -### When *not* to use unit tests - -
- non-exported functions or classes - Anything that is not exported from a module can be considered private or an implementation detail and doesn't need to be tested. -
- -
- constants - Testing the value of a constant would mean to copy it. - This results in extra effort without additional confidence that the value is correct. -
- -
- Vue components - Computed properties, methods, and lifecycle hooks can be considered an implementation detail of components and don't need to be tested. - They are implicitly covered by component tests. - The official Vue guidelines suggest the same. -
- -### What to mock in unit tests - -
- state of the class under test - Modifying the state of the class under test directly rather than using methods of the class avoids side-effects in test setup. -
- -
- other exported classes - Every class needs to be tested in isolation to prevent test scenarios from growing exponentially. -
- -
- single DOM elements if passed as parameters - For tests that only operate on single DOM elements rather than a whole page, creating these elements is cheaper than loading a whole HTML fixture. -
- -
- all server requests - When running frontend unit tests, the backend may not be reachable. - Therefore all outgoing requests need to be mocked. -
- -
- asynchronous background operations - Background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. -
- -### What *not* to mock in unit tests - -
- non-exported functions or classes - Everything that is not exported can be considered private to the module and will be implicitly tested via the exported classes / functions. -
- -
- methods of the class under test - By mocking methods of the class under test, the mocks will be tested and not the real methods. -
- -
- utility functions (pure functions, or those that only modify parameters) - If a function has no side effects because it has no state, it is safe to not mock it in tests. -
- -
- full HTML pages - Loading the HTML of a full page slows down tests, so it should be avoided in unit tests. -
- -## Frontend component tests - -Component tests cover the state of a single component that is perceivable by a user depending on external signals such as user input, events fired from other components, or application state. - -### When to use component tests - -- Vue components - -### When *not* to use component tests - -
- Vue applications - Vue applications may contain many components. - Testing them on a component level requires too much effort. - Therefore they are tested on frontend integration level. -
- -
- HAML templates - HAML templates contain only Markup and no frontend-side logic. - Therefore they are not complete components. -
- -### What to mock in component tests - -
- DOM - Operating on the real DOM is significantly slower than on the virtual DOM. -
- -
- properties and state of the component under test - Similarly to testing classes, modifying the properties directly (rather than relying on methods of the component) avoids side-effects. -
- -
- Vuex store - To avoid side effects and keep component tests simple, Vuex stores are replaced with mocks. -
- -
- all server requests - Similar to unit tests, when running component tests, the backend may not be reachable. - Therefore all outgoing requests need to be mocked. -
- -
- asynchronous background operations - Similar to unit tests, background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. -
- -
- child components - Every component is tested individually, so child components are mocked. - See also shallowMount() -
- -### What *not* to mock in component tests - -
- methods or computed properties of the component under test - By mocking part of the component under test, the mocks will be tested and not the real component. -
- -
- functions and classes independent from Vue - All plain JavaScript code is already covered by unit tests and needs not to be mocked in component tests. -
- -## Frontend integration tests - -Integration tests cover the interaction between all components on a single page. -Their abstraction level is comparable to how a user would interact with the UI. - -### When to use integration tests - -
- page bundles (index.js files in app/assets/javascripts/pages/) - Testing the page bundles ensures the corresponding frontend components integrate well. -
- -
- Vue applications outside of page bundles - Testing Vue applications as a whole ensures the corresponding frontend components integrate well. -
- -### What to mock in integration tests - -
- HAML views (use fixtures instead) - Rendering HAML views requires a Rails environment including a running database which we cannot rely on in frontend tests. -
- -
- all server requests - Similar to unit and component tests, when running component tests, the backend may not be reachable. - Therefore all outgoing requests need to be mocked. -
- -
- asynchronous background operations that are not perceivable on the page - Background operations that affect the page need to be tested on this level. - All other background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. -
- -### What *not* to mock in integration tests - -
- DOM - Testing on the real DOM ensures our components work in the environment they are meant for. - Part of this will be delegated to cross-browser testing. -
- -
- properties or state of components - On this level, all tests can only perform actions a user would do. - For example to change the state of a component, a click event would be fired. -
- -
- Vuex stores - When testing the frontend code of a page as a whole, the interaction between Vue components and Vuex stores is covered as well. -
- -## Feature tests - -In contrast to [frontend integration tests](#frontend-integration-tests), feature tests make requests against the real backend instead of using fixtures. -This also implies that database queries are executed which makes this category significantly slower. - -See also the [RSpec testing guidelines](../../testing_guide/best_practices.md#rspec). - -### When to use feature tests - -- use cases that require a backend and cannot be tested using fixtures -- behavior that is not part of a page bundle but defined globally - -### Relevant notes - -A `:js` flag is added to the test to make sure the full environment is loaded. - -``` -scenario 'successfully', :js do - sign_in(create(:admin)) -end -``` - -The steps of each test are written using capybara methods ([documentation](https://www.rubydoc.info/gems/capybara)). - -Bear in mind XHR calls might require you to use `wait_for_requests` in between steps, like so: - -```rspec -find('.form-control').native.send_keys(:enter) - -wait_for_requests - -expect(page).not_to have_selector('.card') -``` - -## Test helpers - -### Vuex Helper: `testAction` - -We have a helper available to make testing actions easier, as per [official documentation](https://vuex.vuejs.org/guide/testing.html): - -``` -testAction( - actions.actionName, // action - { }, // params to be passed to action - state, // state - [ - { type: types.MUTATION}, - { type: types.MUTATION_1, payload: {}}, - ], // mutations committed - [ - { type: 'actionName', payload: {}}, - { type: 'actionName1', payload: {}}, - ] // actions dispatched - done, -); -``` - -Check an example in [spec/javascripts/ide/stores/actions_spec.jsspec/javascripts/ide/stores/actions_spec.js](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/javascripts/ide/stores/actions_spec.js). - -### Vue Helper: `mountComponent` - -To make mounting a Vue component easier and more readable, we have a few helpers available in `spec/helpers/vue_mount_component_helper`. - -- `createComponentWithStore` -- `mountComponentWithStore` - -Examples of usage: - -``` -beforeEach(() => { - vm = createComponentWithStore(Component, store); - - vm.$store.state.currentBranchId = 'master'; - - vm.$mount(); -}, -``` - -``` -beforeEach(() => { - vm = mountComponentWithStore(Component, { - el: '#dummy-element', - store, - props: { badge }, - }); -}, -``` - -Don't forget to clean up: - -``` -afterEach(() => { - vm.$destroy(); -}); -``` - -## Testing with older browsers - -Some regressions only affect a specific browser version. We can install and test in particular browsers with either Firefox or Browserstack using the following steps: - -### Browserstack - -[Browserstack](https://www.browserstack.com/) allows you to test more than 1200 mobile devices and browsers. -You can use it directly through the [live app](https://www.browserstack.com/live) or you can install the [chrome extension](https://chrome.google.com/webstore/detail/browserstack/nkihdmlheodkdfojglpcjjmioefjahjb) for easy access. -You can find the credentials on 1Password, under `frontendteam@gitlab.com`. - -### Firefox - -#### macOS - -You can download any older version of Firefox from the releases FTP server, - -1. From the website, select a version, in this case `50.0.1`. -1. Go to the mac folder. -1. Select your preferred language, you will find the dmg package inside, download it. -1. Drag and drop the application to any other folder but the `Applications` folder. -1. Rename the application to something like `Firefox_Old`. -1. Move the application to the `Applications` folder. -1. Open up a terminal and run `/Applications/Firefox_Old.app/Contents/MacOS/firefox-bin -profilemanager` to create a new profile specific to that Firefox version. -1. Once the profile has been created, quit the app, and run it again like normal. You now have a working older Firefox version. diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 2985278cc92..52ac746716c 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -593,6 +593,365 @@ end [capybara]: https://github.com/teamcapybara/capybara [jasmine]: https://jasmine.github.io/ +## Overview of Frontend Testing Levels + +Tests relevant for frontend development can be found at the following places: + +- `spec/javascripts/` which are run by Karma (command: `yarn karma`) and contain + - [frontend unit tests](#frontend-unit-tests) + - [frontend component tests](#frontend-component-tests) + - [frontend integration tests](#frontend-integration-tests) +- `spec/frontend/` which are run by Jest (command: `yarn jest`) and contain + - [frontend unit tests](#frontend-unit-tests) + - [frontend component tests](#frontend-component-tests) + - [frontend integration tests](#frontend-integration-tests) +- `spec/features/` which are run by RSpec and contain + - [feature tests](#feature-tests) + +All tests in `spec/javascripts/` will eventually be migrated to `spec/frontend/` (see also [#52483](https://gitlab.com/gitlab-org/gitlab-ce/issues/52483)). + +In addition, there used to be feature tests in `features/`, run by Spinach. +These were removed from the codebase in May 2018 ([#23036](https://gitlab.com/gitlab-org/gitlab-ce/issues/23036)). + +See also [Notes on testing Vue components](../fe_guide/vue.html#testing-vue-components). + +### Frontend unit tests + +Unit tests are on the lowest abstraction level and typically test functionality that is not directly perceivable by a user. + +#### When to use unit tests + +
+ exported functions and classes + Anything that is exported can be reused at various places in a way you have no control over. + Therefore it is necessary to document the expected behavior of the public interface with tests. +
+ +
+ Vuex actions + Any Vuex action needs to work in a consistent way independent of the component it is triggered from. +
+ +
+ Vuex mutations + For complex Vuex mutations it helps to identify the source of a problem by separating the tests from other parts of the Vuex store. +
+ +#### When *not* to use unit tests + +
+ non-exported functions or classes + Anything that is not exported from a module can be considered private or an implementation detail and doesn't need to be tested. +
+ +
+ constants + Testing the value of a constant would mean to copy it. + This results in extra effort without additional confidence that the value is correct. +
+ +
+ Vue components + Computed properties, methods, and lifecycle hooks can be considered an implementation detail of components and don't need to be tested. + They are implicitly covered by component tests. + The official Vue guidelines suggest the same. +
+ +#### What to mock in unit tests + +
+ state of the class under test + Modifying the state of the class under test directly rather than using methods of the class avoids side-effects in test setup. +
+ +
+ other exported classes + Every class needs to be tested in isolation to prevent test scenarios from growing exponentially. +
+ +
+ single DOM elements if passed as parameters + For tests that only operate on single DOM elements rather than a whole page, creating these elements is cheaper than loading a whole HTML fixture. +
+ +
+ all server requests + When running frontend unit tests, the backend may not be reachable. + Therefore all outgoing requests need to be mocked. +
+ +
+ asynchronous background operations + Background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. +
+ +#### What *not* to mock in unit tests + +
+ non-exported functions or classes + Everything that is not exported can be considered private to the module and will be implicitly tested via the exported classes / functions. +
+ +
+ methods of the class under test + By mocking methods of the class under test, the mocks will be tested and not the real methods. +
+ +
+ utility functions (pure functions, or those that only modify parameters) + If a function has no side effects because it has no state, it is safe to not mock it in tests. +
+ +
+ full HTML pages + Loading the HTML of a full page slows down tests, so it should be avoided in unit tests. +
+ +### Frontend component tests + +Component tests cover the state of a single component that is perceivable by a user depending on external signals such as user input, events fired from other components, or application state. + +#### When to use component tests + +- Vue components + +#### When *not* to use component tests + +
+ Vue applications + Vue applications may contain many components. + Testing them on a component level requires too much effort. + Therefore they are tested on frontend integration level. +
+ +
+ HAML templates + HAML templates contain only Markup and no frontend-side logic. + Therefore they are not complete components. +
+ +#### What to mock in component tests + +
+ DOM + Operating on the real DOM is significantly slower than on the virtual DOM. +
+ +
+ properties and state of the component under test + Similarly to testing classes, modifying the properties directly (rather than relying on methods of the component) avoids side-effects. +
+ +
+ Vuex store + To avoid side effects and keep component tests simple, Vuex stores are replaced with mocks. +
+ +
+ all server requests + Similar to unit tests, when running component tests, the backend may not be reachable. + Therefore all outgoing requests need to be mocked. +
+ +
+ asynchronous background operations + Similar to unit tests, background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. +
+ +
+ child components + Every component is tested individually, so child components are mocked. + See also shallowMount() +
+ +#### What *not* to mock in component tests + +
+ methods or computed properties of the component under test + By mocking part of the component under test, the mocks will be tested and not the real component. +
+ +
+ functions and classes independent from Vue + All plain JavaScript code is already covered by unit tests and needs not to be mocked in component tests. +
+ +### Frontend integration tests + +Integration tests cover the interaction between all components on a single page. +Their abstraction level is comparable to how a user would interact with the UI. + +#### When to use integration tests + +
+ page bundles (index.js files in app/assets/javascripts/pages/) + Testing the page bundles ensures the corresponding frontend components integrate well. +
+ +
+ Vue applications outside of page bundles + Testing Vue applications as a whole ensures the corresponding frontend components integrate well. +
+ +#### What to mock in integration tests + +
+ HAML views (use fixtures instead) + Rendering HAML views requires a Rails environment including a running database which we cannot rely on in frontend tests. +
+ +
+ all server requests + Similar to unit and component tests, when running component tests, the backend may not be reachable. + Therefore all outgoing requests need to be mocked. +
+ +
+ asynchronous background operations that are not perceivable on the page + Background operations that affect the page need to be tested on this level. + All other background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. +
+ +#### What *not* to mock in integration tests + +
+ DOM + Testing on the real DOM ensures our components work in the environment they are meant for. + Part of this will be delegated to cross-browser testing. +
+ +
+ properties or state of components + On this level, all tests can only perform actions a user would do. + For example to change the state of a component, a click event would be fired. +
+ +
+ Vuex stores + When testing the frontend code of a page as a whole, the interaction between Vue components and Vuex stores is covered as well. +
+ +### Feature tests + +In contrast to [frontend integration tests](#frontend-integration-tests), feature tests make requests against the real backend instead of using fixtures. +This also implies that database queries are executed which makes this category significantly slower. + +See also the [RSpec testing guidelines](../testing_guide/best_practices.md#rspec). + +#### When to use feature tests + +- Use cases that require a backend and cannot be tested using fixtures. +- Behavior that is not part of a page bundle but defined globally. + +#### Relevant notes + +A `:js` flag is added to the test to make sure the full environment is loaded. + +```js +scenario 'successfully', :js do + sign_in(create(:admin)) +end +``` + +The steps of each test are written using capybara methods ([documentation](https://www.rubydoc.info/gems/capybara)). + +Bear in mind XHR calls might require you to use `wait_for_requests` in between steps, like so: + +```rspec +find('.form-control').native.send_keys(:enter) + +wait_for_requests + +expect(page).not_to have_selector('.card') +``` + +## Test helpers + +### Vuex Helper: `testAction` + +We have a helper available to make testing actions easier, as per [official documentation](https://vuex.vuejs.org/guide/testing.html): + +```js +testAction( + actions.actionName, // action + { }, // params to be passed to action + state, // state + [ + { type: types.MUTATION}, + { type: types.MUTATION_1, payload: {}}, + ], // mutations committed + [ + { type: 'actionName', payload: {}}, + { type: 'actionName1', payload: {}}, + ] // actions dispatched + done, +); +``` + +Check an example in [spec/javascripts/ide/stores/actions_spec.jsspec/javascripts/ide/stores/actions_spec.js](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/javascripts/ide/stores/actions_spec.js). + +### Vue Helper: `mountComponent` + +To make mounting a Vue component easier and more readable, we have a few helpers available in `spec/helpers/vue_mount_component_helper`: + +- `createComponentWithStore` +- `mountComponentWithStore` + +Examples of usage: + +```js +beforeEach(() => { + vm = createComponentWithStore(Component, store); + + vm.$store.state.currentBranchId = 'master'; + + vm.$mount(); +}); +``` + +```js +beforeEach(() => { + vm = mountComponentWithStore(Component, { + el: '#dummy-element', + store, + props: { badge }, + }); +}); +``` + +Don't forget to clean up: + +```js +afterEach(() => { + vm.$destroy(); +}); +``` + +## Testing with older browsers + +Some regressions only affect a specific browser version. We can install and test in particular browsers with either Firefox or Browserstack using the following steps: + +### Browserstack + +[Browserstack](https://www.browserstack.com/) allows you to test more than 1200 mobile devices and browsers. +You can use it directly through the [live app](https://www.browserstack.com/live) or you can install the [chrome extension](https://chrome.google.com/webstore/detail/browserstack/nkihdmlheodkdfojglpcjjmioefjahjb) for easy access. +You can find the credentials on 1Password, under `frontendteam@gitlab.com`. + +### Firefox + +#### macOS + +You can download any older version of Firefox from the releases FTP server, : + +1. From the website, select a version, in this case `50.0.1`. +1. Go to the mac folder. +1. Select your preferred language, you will find the dmg package inside, download it. +1. Drag and drop the application to any other folder but the `Applications` folder. +1. Rename the application to something like `Firefox_Old`. +1. Move the application to the `Applications` folder. +1. Open up a terminal and run `/Applications/Firefox_Old.app/Contents/MacOS/firefox-bin -profilemanager` to create a new profile specific to that Firefox version. +1. Once the profile has been created, quit the app, and run it again like normal. You now have a working older Firefox version. + --- [Return to Testing documentation](index.md)