diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 917214cd1b..c63333e804 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -155,10 +155,7 @@ Fork the repository and make changes on your fork in a feature branch: your intentions, and name it XXXX-something where XXXX is the number of the issue. -Submit unit tests for your changes. Go has a great test framework built in; use -it! Take a look at existing tests for inspiration. [Run the full test -suite](https://docs.docker.com/opensource/project/test-and-docs/) on your branch before -submitting a pull request. +Submit tests for your changes. See [TESTING.md](./TESTING.md) for details. Update the documentation when creating or modifying features. Test your documentation changes for clarity, concision, and correctness, as well as a @@ -251,10 +248,9 @@ calling it in another file constitute a single logical unit of work. The very high majority of submissions should have a single commit, so if in doubt: squash down to one. -After every commit, [make sure the test suite passes] -(https://docs.docker.com/opensource/project/test-and-docs/). Include documentation -changes in the same pull request so that a revert would remove all traces of -the feature or fix. +After every commit, [make sure the test suite passes](./TESTING.md). Include +documentation changes in the same pull request so that a revert would remove +all traces of the feature or fix. Include an issue reference like `Closes #XXXX` or `Fixes #XXXX` in commits that close an issue. Including references automatically closes the issue on a merge. diff --git a/TESTING.md b/TESTING.md new file mode 100644 index 0000000000..20f7c9254e --- /dev/null +++ b/TESTING.md @@ -0,0 +1,71 @@ +# Testing + +This document contains the Moby code testing guidelines. It should answer any +questions you may have as an aspiring Moby contributor. + +## Test suites + +Moby has two test suites (and one legacy test suite): + +* Unit tests - use standard `go test` and + [testify](https://github.com/stretchr/testify) assertions. They are located in + the package they test. Unit tests should be fast and test only their own + package. +* API integration tests - use standard `go test` and + [testify](https://github.com/stretchr/testify) assertions. They are located in + `./integration/` directories, where `component` is: container, + image, volume, etc. These tests perform HTTP requests to an API endpoint and + check the HTTP response and daemon state after the call. + +The legacy test suite `integration-cli/` is deprecated. No new tests will be +added to this suite. Any tests in this suite which require updates should be +ported to either the unit test suite or the new API integration test suite. + +## Writing new tests + +Most code changes will fall into one of the following categories. + +### Writing tests for new features + +New code should be covered by unit tests. If the code is difficult to test with +a unit tests then that is a good sign that it should be refactored to make it +easier to reuse and maintain. Consider accepting unexported interfaces instead +of structs so that fakes can be provided for dependencies. + +If the new feature includes a completely new API endpoint then a new API +integration test should be added to cover the success case of that endpoint. + +If the new feature does not include a completely new API endpoint consider +adding the new API fields to the existing test for that endpoint. A new +integration test should **not** be added for every new API field or API error +case. Error cases should be handled by unit tests. + +### Writing tests for bug fixes + +Bugs fixes should include a unit test case which exercises the bug. + +A bug fix may also include new assertions in an existing integration tests for the +API endpoint. + +## Running tests + +To run the unit test suite: + +``` +make test-unit +``` + +or `hack/test/unit` from inside a `BINDDIR=. make shell` container or properly +configured environment. + +The following environment variables may be used to run a subset of tests: + +* `TESTDIRS` - paths to directories to be tested, defaults to `./...` +* `TESTFLAGS` - flags passed to `go test`, to run tests which match a pattern + use `TESTFLAGS="-test.run TestNameOrPrefix"` + +To run the integration test suite: + +``` +make test-integration +```