Several improvements to error handling:
- Introduce ImageConfigPullError type, wrapping errors related to
downloading the image configuration blob in schema2. This allows for a
more descriptive error message to be seen by the end user.
- Change some logrus.Debugf calls that display errors to logrus.Errorf.
Add log lines in the push/pull fallback cases to make sure the errors
leading to the fallback are shown.
- Move error-related types and functions which are only used by the
distribution package out of the registry package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This reverts commit 84b2162c1a.
The intent of this commit was to set an idle timeout on a HTTP
connection. If a read took more than 60 seconds to complete, or a write
took more than 60 seconds to complete, the connection would be
considered dead.
This doesn't work properly, because the HTTP internals apparently read
from the connection concurrently while writing. An upload that doesn't
complete in 60 seconds leads to a timeout.
Fixes#19967
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Otherwise, some operations can get stuck indefinitely when the remote
side is unresponsive.
Fixes#12823
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
There was already a check that prevented protocol-level fallback in this
situation, but retries within a specific protocol will still happen.
This makes it take a long time for the pull to finally error out.
This fixes slowness in TestDaemonNoSpaceleftOnDeviceError, which used to
take a long time due to the backoff between retry attempts:
PASS: docker_cli_daemon_test.go:1868: DockerDaemonSuite.TestDaemonNoSpaceleftOnDeviceError 5.882s
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
When authorization errors are returned by the token process the error will be wrapped in url.Error.
In order to check the underlying error for retry this error message should be unwrapped.
Unwrapping this error allows failure to push due to an unauthorized response to keep from retrying, possibly resulting in later 429 responses.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
If we detect a Docker-Distribution-Api-Version header indicating that
the registry speaks the V2 protocol, no fallback to V1 should take
place.
The same applies if a V2 registry operation succeeds while attempting a
push or pull.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This commit adds a transfer manager which deduplicates and schedules
transfers, and also an upload manager and download manager that build on
top of the transfer manager to provide high-level interfaces for uploads
and downloads. The push and pull code is modified to use these building
blocks.
Some benefits of the changes:
- Simplification of push/pull code
- Pushes can upload layers concurrently
- Failed downloads and uploads are retried after backoff delays
- Cancellation is supported, but individual transfers will only be
cancelled if all pushes or pulls using them are cancelled.
- The distribution code is decoupled from Docker Engine packages and API
conventions (i.e. streamformatter), which will make it easier to split
out.
This commit also includes unit tests for the new distribution/xfer
package. The tests cover 87.8% of the statements in the package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This change allows API clients to retrieve an authentication token from
a registry, and then pass that token directly to the API.
Example usage:
REPO_USER=dhiltgen
read -s PASSWORD
REPO=privateorg/repo
AUTH_URL=https://auth.docker.io/token
TOKEN=$(curl -s -u "${REPO_USER}:${PASSWORD}" "${AUTH_URL}?scope=repository:${REPO}:pull&service=registry.docker.io" |
jq -r ".token")
HEADER=$(echo "{\"registrytoken\":\"${TOKEN}\"}"|base64 -w 0 )
curl -s -D - -H "X-Registry-Auth: ${HEADER}" -X POST "http://localhost:2376/images/create?fromImage=${REPO}"
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
