If a build context tar has path names of the form 'x/./y', they will be
stored in this unnormalized form internally by tarsum. When the builder
walks the untarred directory tree and queries hashes for each relative
path, it will query paths of the form 'x/y', and they will not be found.
To correct this, have tarsum normalize path names by calling Clean.
Add a test to detect this caching false positive.
Fixes#21715
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Updates `image.StoreImage()` to always ensure that images
that are installed in Docker have a tarsum.v1 checksum.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
While the v2 pull operation is writing the body of the layer blob to disk
it now computes the tarsum checksum of the archive before extracting it to
the backend storage driver. If the checksum does not match that from the
image manifest an error is raised.
Also adds more debug logging to the pull operation and fixes existing test
cases which were failing. Adds a reverse lookup constructor to the tarsum
package so that you can get a tarsum object using a checksum label.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
The current Dev version of TarSum includes hashing of extended
file attributes and omits inclusion of modified time headers.
I refactored the logic around the version differences to make it
more clear that the difference between versions is in how tar
headers are selected and ordered.
TarSum Version 1 is now declared with the new Dev version continuing
to track it.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
When read is called on a tarsum with a two different read sizes, specifically the second call larger than the first, the dynamic buffer does not get reallocated causing a slice read error.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Not breaking the default cipher and algorithm for calculating checksums,
but allow for using alternate block ciphers during the checksum
calculation.
Docker-DCO-1.1-Signed-off-by: Vincent Batts <vbatts@redhat.com> (github: vbatts)
If a tar were constructed with duplicate file names, then depending on
the order, it could result in same tarsum.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Tarsum now correctly closes the internal TarWriter which appends
a block of 1024 zeros to the end of the returned archive.
Signed-off-by: Josh Hawn <josh.hawn@docker.com>
this is to enhance the tarsum algorithm, but _MUST_ be done in lock step
with the same for docker-registry. (PR will be cited)
Docker-DCO-1.1-Signed-off-by: Vincent Batts <vbatts@redhat.com> (github: vbatts)
This introduces Versions for TarSum checksums.
Fixes: https://github.com/docker/docker/issues/7526
It preserves current functionality and abstracts the interface for
future flexibility of hashing algorithms. As a POC, the VersionDev
Tarsum does not include the mtime in the checksum calculation, and would
solve https://github.com/docker/docker/issues/7387 though this is not a
settled Version is subject to change until a version number is assigned.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
This commit makes tarsum buffer allocation dynamic. This change
is required to avoid allocating memory excessively after the archive
buffering changes.
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
