This updates the vendored docker/distribution to the current master
branch.
Note the following changes:
- The manifest package was split into manifest/schema1. Most references
to the manifest package in the engine needed to be updated to use
schema1 instead.
- Validation functions in api/v2 were replaced by the
distribution/reference package. The engine code has been updated to
use the reference package for validation where necessary. A future PR
will change the engine to use the types defined in
distribution/reference more comprehensively.
- The reference package explicitly allows double _ characters in
repository names. registry_test.go was updated for this.
- TestPullFailsWithAlteredManifest was corrupting the manifest JSON, now
that the schema1 package unmarshals the correct payload. The test is
being changed to modify the JSON without affecting its length, which
allows the pull to succeed to the point where digest validation
happens.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Also remove unused func `newTempFile` and prevent a possible deadlock
between pull_v2 `attemptIDReuse` and graph `register`
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Ensure v2 registries are given more than 5 seconds to return a ping and avoid an unnecessary fallback to v1.
Elevates log level about failed v2 ping to a warning to match the warning related to using v1 registries.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Add a unit test for validateManifest which ensures extra data can't be
injected by adding data to the JSON object outside the payload area.
This also removes validation of legacy signatures at pull time. This
starts the path of deprecating legacy signatures, whose presence in the
very JSON document they attempt to sign is problematic. These
signatures were only checked for official images, and since they only
caused a weakly-worded message to be printed, removing the verification
should not cause impact.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Generate a hash chain involving the image configuration, layer digests,
and parent image hashes. Use the digests to compute IDs for each image
in a manifest, instead of using the remotely specified IDs.
To avoid breaking users' caches, check for images already in the graph
under old IDs, and avoid repulling an image if the version on disk under
the legacy ID ends up with the same digest that was computed from the
manifest for that image.
When a calculated ID already exists in the graph but can't be verified,
continue trying SHA256(digest) until a suitable ID is found.
"save" and "load" are not changed to use a similar scheme. "load" will
preserve the IDs present in the tar file.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
progressreader.Broadcaster becomes broadcaster.Buffered and
broadcastwriter.Writer becomes broadcaster.Unbuffered.
The package broadcastwriter is thus renamed to broadcaster.
Signed-off-by: Tibor Vass <tibor@docker.com>
The original purpose of this was to cancel downloads if pullV2Tag
returns an error, preventing an associated crash (see #15353). The
broadcaster now accomplishes the same thing that the pipe does, making
the pipe redundant. When pullV2Tag returns, all broadcasters are closed,
which means all further writes to those broadcasters will return errors.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Before, this only waited for the download to complete. There was no
guarantee that the layer had been registered in the graph and was ready
use. This is especially problematic with v2 pulls, which wait for all
downloads before extracting layers.
Change Broadcaster to allow an error value to be propagated from Close
to the waiters.
Make the wait stop when the extraction is finished, rather than just the
download.
This also fixes v2 layer downloads to prefix the pool key with "layer:"
instead of "img:". "img:" is the wrong prefix, because this is what v1
uses for entire images. A v1 pull waiting for one of these operations to
finish would only wait for that particular layer, not all its
dependencies.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The Docker Daemon should send actual actions client ask for to issue tokens,
not all the permissions that client is guaranteed.
Signed-off-by: xiekeyang <xiekeyang@huawei.com>
- Rename to Broadcaster
- Document exported types
- Change Wait function to just wait. Writing a message to the writer and
adding the writer to the observers list are now handled by separate
function calls.
- Avoid importing logrus (the condition where it was used should never
happen, anyway).
- Make writes non-blocking
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Previously, its other return value was used even when it returned an
error. This is awkward and goes against the convention. It also could
have resulted in a nil pointer dereference when an error was returned
because of an unknown pool type. This changes the unknown pool type
error to a panic (since the pool types are hardcoded at call sites and
must always be "push" or "pull"), and returns a "found" boolean instead
of an error.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Based on #12874 from Sam Abed <sam.abed@gmail.com>. His original commit
was brought up to date by manually porting the changes in pull.go into
the new code in pull_v1.go and pull_v2.go.
Fixes#8385
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
* use downloadInfo pointers everywhere
* use downloads slice only for things that we really download
* cleanup tmp files in all cases
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
The process of pulling an image spawns a new goroutine for each layer in the
image manifest. If any of these downloads fail we would stop everything and
return the error, even though other goroutines would still be running and
writing output through a progress reader which is attached to an http response
writer. Since the request handler had already returned from the first error,
the http server panics when one of these download goroutines makes a write to
the response writer buffer.
This patch prevents this crash in the daemon http server by waiting for all of
the download goroutines to complete, even if one of them fails. Only then does
it return, terminating the request handler.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
We noticed a regression since the 1.7.1 patch after some refactoring. This
patch corrects the behavior and adds integration tests for modified manifest
and rootfs layer blobs.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Some structures use int for sizes and UNIX timestamps. On some
platforms, int is 32 bits, so this can lead to the year 2038 issues and
overflows when dealing with large containers or layers.
Consistently use int64 to store sizes and UNIX timestamps in
api/types/types.go. Update related to code accordingly (i.e.
strconv.FormatInt instead of strconv.Itoa).
Use int64 in progressreader package to avoid integer overflow when
dealing with large quantities. Update related code accordingly.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Add a trusted flag to force the cli to resolve a tag into a digest via the notary trust library and pull by digest.
On push the flag the trust flag will indicate the digest and size of a manifest should be signed and push to a notary server.
If a tag is given, the cli will resolve the tag into a digest and pull by digest.
After pulling, if a tag is given the cli makes a request to tag the image.
Use certificate directory for notary requests
Read certificates using same logic used by daemon for registry requests.
Catch JSON syntax errors from Notary client
When an uncaught error occurs in Notary it may show up in Docker as a JSON syntax error, causing a confusing error message to the user.
Provide a generic error when a JSON syntax error occurs.
Catch expiration errors and wrap in additional context.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
