1
0
Fork 0
mirror of https://github.com/moby/moby.git synced 2022-11-09 12:21:53 -05:00
Moby Project - a collaborative project for the container ecosystem to assemble container-based systems
Find a file
Sebastiaan van Stijn 13cff6d583
Fix race in TestCreateServiceSecretFileMode, TestCreateServiceConfigFileMode
Looks like this test was broken from the start, and fully relied on a race
condition. (Test was added in 65ee7fff02)

The problem is in the service's command: `ls -l /etc/config || /bin/top`, which
will either:

- exit immediately if the secret is mounted correctly at `/etc/config` (which it should)
- keep running with `/bin/top` if the above failed

After the service is created, the test enters a race-condition, checking for 1
task to be running (which it ocassionally is), after which it proceeds, and looks
up the list of tasks of the service, to get the log output of `ls -l /etc/config`.

This is another race: first of all, the original filter for that task lookup did
not filter by `running`, so it would pick "any" task of the service (either failed,
running, or "completed" (successfully exited) tasks).

In the meantime though, SwarmKit kept reconciling the service, and creating new
tasks, so even if the test was able to get the ID of the correct task, that task
may already have been exited, and removed (task-limit is 5 by default), so only
if the test was "lucky", it would be able to get the logs, but of course, chances
were likely that it would be "too late", and the task already gone.

The problem can be easily reproduced when running the steps manually:

    echo 'CONFIG' | docker config create myconfig -

    docker service create --config source=myconfig,target=/etc/config,mode=0777 --name myservice busybox sh -c 'ls -l /etc/config || /bin/top'

The above creates the service, but it keeps retrying, because each task exits
immediately (followed by SwarmKit reconciling and starting a new task);

    mjntpfkkyuuc1dpay4h00c4oo
    overall progress: 0 out of 1 tasks
    1/1: ready     [======================================>            ]
    verify: Detected task failure
    ^COperation continuing in background.
    Use `docker service ps mjntpfkkyuuc1dpay4h00c4oo` to check progress.

And checking the tasks for the service reveals that tasks exit cleanly (no error),
but _do exit_, so swarm just keeps up reconciling, and spinning up new tasks;

    docker service ps myservice --no-trunc
    ID                          NAME              IMAGE                                                                                    NODE             DESIRED STATE   CURRENT STATE                     ERROR     PORTS
    2wmcuv4vffnet8nybg3he4v9n   myservice.1       busybox:latest@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57   docker-desktop   Ready           Ready less than a second ago
    5p8b006uec125iq2892lxay64    \_ myservice.1   busybox:latest@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57   docker-desktop   Shutdown        Complete less than a second ago
    k8lpsvlak4b3nil0zfkexw61p    \_ myservice.1   busybox:latest@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57   docker-desktop   Shutdown        Complete 6 seconds ago
    vsunl5pi7e2n9ol3p89kvj6pn    \_ myservice.1   busybox:latest@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57   docker-desktop   Shutdown        Complete 11 seconds ago
    orxl8b6kt2l6dfznzzd4lij4s    \_ myservice.1   busybox:latest@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57   docker-desktop   Shutdown        Complete 17 seconds ago

This patch changes the service's command to `sleep`, so that a successful task
(after successfully performing `ls -l /etc/config`) continues to be running until
the service is deleted. With that change, the service should (usually) reconcile
immediately, which removes the race condition, and should also make it faster :)

This patch changes the tests to use client.ServiceLogs() instead of using the
service's tasklist to directly access container logs. This should also fix some
failures that happened if some tasks failed to start before reconciling, in which
case client.TaskList() (with the current filters), could return more tasks than
anticipated (as it also contained the exited tasks);

    === RUN   TestCreateServiceSecretFileMode
        create_test.go:291: assertion failed: 2 (int) != 1 (int)
    --- FAIL: TestCreateServiceSecretFileMode (7.88s)
    === RUN   TestCreateServiceConfigFileMode
        create_test.go:355: assertion failed: 2 (int) != 1 (int)
    --- FAIL: TestCreateServiceConfigFileMode (7.87s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-27 10:55:54 +02:00
.github Remove LCOW code (step 1) 2021-06-03 21:16:21 +02:00
api Makefile: update bfirsh/redoc to 1.14.0, and fix swagger warnings 2021-09-24 17:23:55 +02:00
builder buildkit: normalize build target and local platform 2021-10-19 17:11:06 +02:00
cli remove uses of deprecated pkg/term 2020-04-21 16:29:27 +02:00
client Merge pull request #42770 from thaJeztah/eventtype_enums 2021-08-28 00:23:56 +09:00
cmd Allow switching Windows runtimes. 2021-09-23 17:44:04 +00:00
container refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
contrib dockerd-rootless.sh: Fix variable not double quotes cause unexpected behavior 2021-10-18 00:11:03 +08:00
daemon daemon: fix TestVerifyPlatformContainerResources not capturing variable 2021-10-20 09:57:54 +02:00
distribution Merge pull request #42782 from dkkb/McGrady/code_improvement 2021-08-30 13:19:08 +02:00
dockerversion Update to Go 1.17.0, and gofmt with Go 1.17 2021-08-24 23:33:27 +02:00
docs docs/api: add node about concurrency on /system/df endpoint 2021-08-10 14:51:27 +02:00
errdefs Implement Unwrap to errors 2020-11-21 16:36:35 +01:00
hack CI: update docker-py to 5.0.3 2021-10-20 09:57:11 +02:00
image refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
integration Fix race in TestCreateServiceSecretFileMode, TestCreateServiceConfigFileMode 2021-10-27 10:55:54 +02:00
integration-cli refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
internal/test/suite integration-cli: move each test suite to its own TestX testing function 2019-09-18 18:26:36 +00:00
layer refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
libcontainerd refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
libnetwork fix TestInvalidRemoteDriver() to check underlying error 2021-10-25 14:57:18 +02:00
oci oci/caps: limit available capabilities to current environment 2021-10-15 16:12:26 +02:00
opts Update to Go 1.17.0, and gofmt with Go 1.17 2021-08-24 23:33:27 +02:00
patches update archive/tar patch for go 1.16 2021-06-10 13:04:48 +02:00
pkg Merge pull request #41215 from cpuguy83/better_plugin_errors 2021-10-24 12:37:07 +09:00
plugin refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
profiles refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
project Fix a typo in project/GOVERANCE.md 2021-08-23 11:15:51 +08:00
quota refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
reference refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
registry Merge pull request #42915 from thaJeztah/registry_cleanup 2021-10-15 10:01:22 +02:00
reports Fix typos 2018-05-16 09:15:43 +08:00
restartmanager unconvert: remove unnescessary conversions 2019-09-18 12:57:33 +02:00
rootless refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
runconfig refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
testutil Port testutil/daemon to FreeBSD 2021-10-16 09:51:55 +03:00
vendor Merge pull request #42933 from thaJeztah/limit_caps_to_environment 2021-10-20 09:55:08 +02:00
volume refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
.DEREK.yml Update .DEREK.yml 2019-05-06 13:27:04 -07:00
.dockerignore update .dockerignore to prevent '-unsupported' builds 2020-01-13 12:55:15 +01:00
.gitignore Use -X ldflags to set dockerversion package vars 2019-12-02 10:21:41 -08:00
.mailmap Update AUTHORS and mailmap 2021-06-04 02:18:49 +02:00
AUTHORS Update AUTHORS and mailmap 2021-06-04 02:18:49 +02:00
CHANGELOG.md changelog: fix incorrectly named libnetwork label 2021-05-31 15:27:46 +02:00
codecov.yml Add code coverage report and codecov config 2018-01-16 16:50:56 -05:00
CONTRIBUTING.md Update CONTRIBUTING.md to have an option to keep name anonymous if requested 2019-07-15 16:04:11 +02:00
Dockerfile Dockerfile: switch CRIU install to Debian 11 "bullseye" packages 2021-10-15 10:43:15 +02:00
Dockerfile.e2e Update Go to 1.17.2 2021-10-08 15:12:36 +02:00
Dockerfile.simple Update Go to 1.17.2 2021-10-08 15:12:36 +02:00
Dockerfile.windows Update containerd binary to v1.5.7 2021-10-15 12:48:29 +02:00
Jenkinsfile Merge pull request #42765 from thaJeztah/windows_2022_ltsc 2021-08-27 15:52:58 +02:00
LICENSE Update LICENSE 2018-09-12 14:27:53 +01:00
MAINTAINERS Add Samuel Karp as a maintainer 2021-05-20 12:54:40 -07:00
Makefile Merge pull request #42880 from thaJeztah/makefile_proxy_vars 2021-09-27 13:30:39 +02:00
NOTICE switch kr/pty to creack/pty v1.1.7 2019-07-29 16:59:08 -07:00
poule.yml Poule: remove random assign 2019-09-05 20:57:17 +02:00
README.md Add "Lego set" back in README.md 2017-10-10 14:10:39 +00:00
ROADMAP.md Fix some typos in ROADMAP.md 2019-01-25 14:27:13 +08:00
SECURITY.md Update SECURITY.md to have an option to keep name anonymous if requested. 2019-06-18 16:37:16 +00:00
TESTING.md TESTING.md: note that integration-cli is deprecated 2020-12-18 07:51:46 +01:00
vendor.conf Merge pull request #42933 from thaJeztah/limit_caps_to_environment 2021-10-20 09:55:08 +02:00
VENDORING.md

The Moby Project

Moby Project logo

Moby is an open-source project created by Docker to enable and accelerate software containerization.

It provides a "Lego set" of toolkit components, the framework for assembling them into custom container-based systems, and a place for all container enthusiasts and professionals to experiment and exchange ideas. Components include container build tools, a container registry, orchestration tools, a runtime and more, and these can be used as building blocks in conjunction with other tools and projects.

Principles

Moby is an open project guided by strong principles, aiming to be modular, flexible and without too strong an opinion on user experience. It is open to the community to help set its direction.

  • Modular: the project includes lots of components that have well-defined functions and APIs that work together.
  • Batteries included but swappable: Moby includes enough components to build fully featured container system, but its modular architecture ensures that most of the components can be swapped by different implementations.
  • Usable security: Moby provides secure defaults without compromising usability.
  • Developer focused: The APIs are intended to be functional and useful to build powerful tools. They are not necessarily intended as end user tools but as components aimed at developers. Documentation and UX is aimed at developers not end users.

Audience

The Moby Project is intended for engineers, integrators and enthusiasts looking to modify, hack, fix, experiment, invent and build systems based on containers. It is not for people looking for a commercially supported system, but for people who want to work and learn with open source code.

Relationship with Docker

The components and tools in the Moby Project are initially the open source components that Docker and the community have built for the Docker Project. New projects can be added if they fit with the community goals. Docker is committed to using Moby as the upstream for the Docker Product. However, other projects are also encouraged to use Moby as an upstream, and to reuse the components in diverse ways, and all these uses will be treated in the same way. External maintainers and contributors are welcomed.

The Moby project is not intended as a location for support or feature requests for Docker products, but as a place for contributors to work on open source code, fix bugs, and make the code more useful. The releases are supported by the maintainers, community and users, on a best efforts basis only, and are not intended for customers who want enterprise or commercial support; Docker EE is the appropriate product for these use cases.


Legal

Brought to you courtesy of our legal counsel. For more context, please see the NOTICE document in this repo.

Use and transfer of Moby may be subject to certain restrictions by the United States and other governments.

It is your responsibility to ensure that your use and/or transfer does not violate applicable laws.

For more information, please see https://www.bis.doc.gov

Licensing

Moby is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.