1
0
Fork 0
mirror of https://github.com/moby/moby.git synced 2022-11-09 12:21:53 -05:00
Commit graph

37113 commits

Author SHA1 Message Date
Sebastiaan van Stijn
3c15cea650
docker-py: use host-network for nested build of docker-py
When building this image docker-in-docker, the DNS in the environment
may not be usable for the build-container, causing resolution to fail:

```
02:35:31 W: Failed to fetch http://deb.debian.org/debian/dists/jessie/Release.gpg  Temporary failure resolving 'deb.debian.org'
```

This patch detects if we're building from within a container, and if
so, skips creating a networking namespace for the build by using
`--network=host`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-16 14:34:54 +02:00
Sebastiaan van Stijn
ba8f4c7994
docker-py: don't build --quiet is TESTDEBUG is set
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-16 14:34:52 +02:00
Sebastiaan van Stijn
968345bc5c
Makefile: Allow passing DOCKER_TEST_HOST and TESTDEBUG to container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-16 14:34:49 +02:00
Sebastiaan van Stijn
7bfe48cc00
Bump docker-py to 4.0.2, and run tests from upstream repository
This removes all the installation steps for docker-py from the
Dockerfile, and instead builds the upstream Dockerfile, and runs
docker-py tests in a container.

To test;

```
make test-docker-py

...

Removing bundles/

---> Making bundle: dynbinary (in bundles/dynbinary)
Building: bundles/dynbinary-daemon/dockerd-dev
Created binary: bundles/dynbinary-daemon/dockerd-dev

---> Making bundle: test-docker-py (in bundles/test-docker-py)
---> Making bundle: .integration-daemon-start (in bundles/test-docker-py)
Using test binary docker
Starting dockerd
INFO: Waiting for daemon to start...
.
INFO: Building docker-sdk-python3:3.7.0...
sha256:686428ae28479e9b5c8fdad1cadc9b7a39b462e66bd13a7e35bd79c6a152a402
INFO: Starting docker-py tests...
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.1.0, py-1.8.0, pluggy-0.9.0
rootdir: /src, inifile: pytest.ini
plugins: timeout-1.3.3, cov-2.6.1
collected 359 items

tests/integration/api_build_test.py .......s....
....
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-16 14:34:46 +02:00
Sebastiaan van Stijn
81dbed4c8b
Merge pull request from thaJeztah/pull_platform_regression
Fix error handling of incorrect --platform values
2019-07-16 03:29:16 +02:00
Sebastiaan van Stijn
2645e31d24
Merge pull request from thaJeztah/remove_codecov
Remove Codecov
2019-07-15 23:54:23 +02:00
Brian Goff
549e34c732
Merge pull request from thaJeztah/errdefs_no_recurse
errdefs: remove unneeded recursive calls
2019-07-15 14:53:40 -07:00
Sebastiaan van Stijn
9d1b4f5fc3
Add regression tests for invalid platform status codes
Before we handled containerd errors, using an invalid platform produced a 500 status:

```bash
curl -v \
  -X POST \
  --unix-socket /var/run/docker.sock \
  "http://localhost:2375/v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest" \
  -H "Content-Type: application/json"
```

```
* Connected to localhost (docker.sock) port 80 ()
> POST /v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest HTTP/1.1
> Host: localhost:2375
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: application/json
>
< HTTP/1.1 500 Internal Server Error
< Api-Version: 1.40
< Content-Length: 85
< Content-Type: application/json
< Date: Mon, 15 Jul 2019 15:25:44 GMT
< Docker-Experimental: true
< Ostype: linux
< Server: Docker/19.03.0-rc2 (linux)
<
{"message":"\"foobar\": unknown operating system or architecture: invalid argument"}
```

That problem is now fixed, and the API correctly returns a 4xx status:

```bash
curl -v \
  -X POST \
  --unix-socket /var/run/docker.sock \
  "http://localhost:2375/v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest" \
  -H "Content-Type: application/json"
```

```
* Connected to localhost (/var/run/docker.sock) port 80 ()
> POST /v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest HTTP/1.1
> Host: localhost:2375
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Type: application/json
>
< HTTP/1.1 400 Bad Request
< Api-Version: 1.41
< Content-Type: application/json
< Docker-Experimental: true
< Ostype: linux
< Server: Docker/dev (linux)
< Date: Mon, 15 Jul 2019 15:13:42 GMT
< Content-Length: 85
<
{"message":"\"foobar\": unknown operating system or architecture: invalid argument"}
* Curl_http_done: called premature == 0
```

This patch adds tests to validate the behaviour

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-15 20:37:00 +02:00
Sebastiaan van Stijn
4a516215e2
errdefs: convert containerd errors to the correct status code
In situations where the containerd error is consumed directly
and not received over gRPC, errors were not translated.

This patch converts containerd errors to the correct HTTP
status code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-15 20:36:57 +02:00
Sebastiaan van Stijn
bd5c5373f1
Remove Codecov
Codecov has shown to be flaky, and calculate the wrong diff, in
addition, it doesn't show coverage for integration tests, which
makes the coverage report not useful.

Removing it for now, while we look at alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-15 20:30:59 +02:00
Brian Goff
ffabf0d542
Merge pull request from thaJeztah/remove_iot_check
Remove skip evaluation of symlinks to data root on IoT Core
2019-07-15 11:13:19 -07:00
Akihiro Suda
cfeba5af9e
Merge pull request from kolyshkin/seccomp-group
profiles/seccomp: improve profile conversion
2019-07-16 03:01:17 +09:00
Akihiro Suda
84edfb216a
Merge pull request from thaJeztah/cleanup_socket_addr
Cleanup "address" when connecting over a UNIX socket
2019-07-16 02:57:30 +09:00
Michael Crosby
78deb9863f
Merge pull request from cpuguy83/better_container_error
Propagate GetContainer error from event processor
2019-07-15 13:53:58 -04:00
Michael Crosby
a7920ecfa1
Merge pull request from thaJeztah/fix_double_host
daemon: don't listen on the same address multiple times
2019-07-15 13:51:09 -04:00
Michael Crosby
3ce23795d7
Merge pull request from cpuguy83/libcontainerd_events_wait
Sleep before restarting event processing
2019-07-15 13:49:05 -04:00
Sebastiaan van Stijn
f51110122f
Merge pull request from thaJeztah/sync_security_reporting
Update CONTRIBUTING.md to have an option to keep name anonymous if requested
2019-07-15 18:30:58 +02:00
Sebastiaan van Stijn
32f4fdfb5c
errdefs: remove unneeded recursive calls
The `statusCodeFromGRPCError` and `statusCodeFromDistributionError`
helpers are used by `GetHTTPErrorStatusCode`, which already recurses
if the error implements the `Causer` interface.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-15 18:22:19 +02:00
Sebastiaan van Stijn
0aa167d0df
Update CONTRIBUTING.md to have an option to keep name anonymous if requested
Same changes as 1d792e6bba, which didn't
update the CONTRIBUTING.md

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-15 16:04:11 +02:00
Sebastiaan van Stijn
6359da4afa
Merge pull request from thaJeztah/macvlan_redundant_checks
Integration: remove redundant kernel version check for MACVlan
2019-07-15 11:33:15 +02:00
Sebastiaan van Stijn
5ce4394486
Merge pull request from selansen/test_initpool
TestServiceWithDefaultAddressPoolInit
2019-07-15 11:25:01 +02:00
selansen
88578aa9e9
TestServiceWithDefaultAddressPoolInit
Looks like TestServiceWithDefaultAddressPoolInit is failing
randomly in CI. I am not able to reproduce the issue locally
but this has been reported few times.  So I tried to modify
code and see if I can fix the random failure.

Signed-off-by: selansen <elango.siva@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-14 20:02:46 +02:00
Sebastiaan van Stijn
f4da929718
Merge pull request from thaJeztah/carry_38654
Fix indentation in some description [carry 38654]
2019-07-14 17:05:36 +02:00
Yong Tang
83ba575d95
Merge pull request from thaJeztah/integration_cleanup_raft
integration: fix cleanup of raft data
2019-07-14 07:42:00 -07:00
Sebastiaan van Stijn
f1b5612f20
Merge pull request from crosbymichael/test-hc-stop
do not stop health check before sending signal
2019-07-14 15:14:13 +02:00
Jérémy Leherpeur
cf315bedc5
Fix indentation in some description
Fix the indentation to allow jane-openapi generate to work

Signed-off-by: Jeremy Leherpeur <jeremy.leherpeur@yousign.fr>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-14 13:42:29 +02:00
Sebastiaan van Stijn
b14313a97a
Merge pull request from skanehira/master
fix swagger.yaml 
2019-07-14 12:24:10 +02:00
Brian Goff
f8aef6a92f
Move kill health test to integration
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-14 11:53:21 +02:00
Ruilin Li
da574f9343
do not stop health check before sending signal
Docker daemon always stops healthcheck before sending signal to a
container now. However, when we use "docker kill" to send signals
other than SIGTERM or SIGKILL to a container, such as SIGINT,
daemon still stops container health check though container process
handles the signal normally and continues to work.

Signed-off-by: Ruilin Li <liruilin4@huawei.com>
2019-07-14 11:53:13 +02:00
Akihiro Suda
e6faa02697
Merge pull request from thaJeztah/stats_early_check
ContainerStats: return early on unsupported API versions
2019-07-14 13:30:34 +09:00
Yong Tang
b3a48b78af
Merge pull request from thaJeztah/lock_portindex
DockerSwarmSuite lock portIndex to work around race
2019-07-13 15:11:27 -07:00
Sebastiaan van Stijn
763454e1e4
Remove unused pkg/system.IsIoTCore()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-13 23:45:08 +02:00
Sebastiaan van Stijn
bad0b4e604
Remove skip evaluation of symlinks to data root on IoT Core
This fix was added in 8e71b1e210 to work around
a go issue (https://github.com/golang/go/issues/20506).

That issue was fixed in
66c03d39f3,
which is part of Go 1.10 and up. This reverts the changes that were made in
8e71b1e210, and are no longer needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-13 23:44:51 +02:00
Sebastiaan van Stijn
44a3267912
Merge pull request from KentaTada/use-memavailable
Use MemAvailable instead of MemFree to estimate actual available memory
2019-07-13 19:59:43 +02:00
Sebastiaan van Stijn
316e16618f
Integration: MACVlan add missing import comment and build-tag
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-13 13:35:21 +02:00
Sebastiaan van Stijn
691eb14256
Integration: remove redundant kernel version check for MACVlan
The daemon requires kernel 3.10 or up to start, so there's no need
to check if the daemon is kernel 3.8 or up.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-13 13:35:18 +02:00
Sebastiaan van Stijn
cfac0c9c49
Merge pull request from thaJeztah/ipvlan_checks_cleanup
Integration: remove redundant checks in IPVLAN tests
2019-07-13 13:27:59 +02:00
Sebastiaan van Stijn
6a64a4deec
integration: fix cleanup of raft data
The directory used for storage was either changed or new directories
were added.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-13 13:23:57 +02:00
Sebastiaan van Stijn
d470252e87
daemon: don't listen on the same address multiple times
Before this change:

    dockerd -H unix:///run/docker.sock -H unix:///run/docker.sock -H unix:///run/docker.sock
    ...
    INFO[2019-07-13T00:02:36.195090937Z] Daemon has completed initialization
    INFO[2019-07-13T00:02:36.215940441Z] API listen on /run/docker.sock
    INFO[2019-07-13T00:02:36.215933172Z] API listen on /run/docker.sock
    INFO[2019-07-13T00:02:36.215990566Z] API listen on /run/docker.sock

After this change:

    dockerd -H unix:///run/docker.sock -H unix:///run/docker.sock -H unix:///run/docker.sock
    ...
    INFO[2019-07-13T00:01:37.533579874Z] Daemon has completed initialization
    INFO[2019-07-13T00:01:37.567045771Z] API listen on /run/docker.sock

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-13 13:21:08 +02:00
Sebastiaan van Stijn
c096225e8e
DockerSwarmSuite lock portIndex to work around race
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-13 13:20:00 +02:00
Sebastiaan van Stijn
47a84dcc64
Merge pull request from cpuguy83/troubleshoot_more_noise
Fix Microsecond -> Millisecond.
2019-07-13 04:10:17 +02:00
Brian Goff
5d818213ff Fix Microsecond -> Milisecond.
A bit too quick on the trigger on some text completion I think...

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-12 18:43:08 -07:00
Brian Goff
1acaf2aabe Sleep before restarting event processing
This prevents restarting event processing in a tight loop.
You can see this with the following steps:

```terminal
$ containerd &
$ dockerd --containerd=/run/containerd/containerd.sock &
$ pkill -9 containerd
```

At this point you will be spammed with logs such as:

```
ERRO[2019-07-12T22:29:37.318761400Z] failed to get event                           error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial unix /run/containerd/containerd.sock: connect: connection refused\"" module=libcontainerd namespace=plugins.moby
```

Without this change you can quickly end up with gigabytes of log data.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-12 15:42:19 -07:00
Sebastiaan van Stijn
0e041d68d8
Merge pull request from thaJeztah/group_test_subdirs
integration: organize bundle directory per test
2019-07-12 21:50:15 +02:00
Kirill Kolyshkin
5cd635b80d
Merge pull request from cpuguy83/test_daemon_ping_logs
Don't log test initial test daemon ping failures
2019-07-12 10:42:40 -07:00
Michael Crosby
a4a1e57e9d
Merge pull request from cpuguy83/fix_missing_dir_cleanup_file
Ensure parent dir exists for mount cleanup file
2019-07-12 13:39:58 -04:00
Michael Crosby
6913656d98
Merge pull request from thaJeztah/harden_TestClientWithRequestTimeout
Harden TestClientWithRequestTimeout
2019-07-12 13:36:36 -04:00
Michael Crosby
1d52c0b82d
Merge pull request from cpuguy83/custom_containerd_namespace
Add (hidden) flags to set containerd namespaces
2019-07-12 13:35:14 -04:00
Sebastiaan van Stijn
8fa0899ff5
Merge pull request from thaJeztah/fix_remove_bundles
Fix "Removing bundles/" not actually removing bundles
2019-07-12 19:17:25 +02:00
Brian Goff
15675e28f1 Don't log test initial test daemon ping failures
This is just noise due to timing. I picked `> 2` just based on
logs from tests I've seen there's always 1 or 2.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-12 09:51:32 -07:00