In commit 3dfc910d77 we changed from
syscall.Chmod() to os.Chmod(), but these take a different form of the
Mode argument. The sycall one takes the raw linux form, wheras
os.Chmod takes the os.FileMode form, and they differ for the higher
bits (setuid, setgid, etc). The raw tar header uses a form which
is compatible with the syscalls, but not the go calls.
We fix this by using hdr.FileInfo() which properly converts the mode
to what go expects.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Files in the .wh..wh.plnk directory are ignored, but other files
inside the tarfile can be hardlinks to these files. This is not
something that normally happens, as on aufs unmount such files are
supposed to be dropped via the "auplink" too, yet images on the index
(such as shipyard/shipyard, e.g. layer
f73c835af6d58b6fc827b400569f79a8f28e54f5bb732be063e1aacefbc374d0)
contains such files.
We handle these by extracting these files to a temporary directory
and resolve such hardlinks via the temporary files.
This fixes https://github.com/dotcloud/docker/issues/3884
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
All archive that are created from somewhere generally have to be closed, because
at some point there is a file or a pipe or something that backs them. So, we
make archive.Archive a ReadCloser. However, code consuming archives does not
typically close them so we add an archive.ArchiveReader and use that when we're
only reading.
We then change all the Tar/Archive places to create ReadClosers, and to properly
close them everywhere.
As an added bonus we can use ReadCloserWrapper rather than EofReader in several places,
which is good as EofReader doesn't always work right. For instance, many compression
schemes like gzip knows it is EOF before having read the EOF from the stream, so the
EofCloser never sees an EOF.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
CompressStream() now always returns a stream that is closable, and it never
closes the underlying writer. TarFilter() makes sure the decompressed stream
is closed at the and, as well as the PipeWriter.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Various kinds of decompressed streams are really ReadClosers. For instance
gzip.NewReader() is, and the one returned from CmdStream is changed to be
because it returns a PipeReader which is a ReadCloser.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This matches what tar does, and without it the tarsum created
by the registry will not match the docker one.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This uses a plain filepath.Walk + addTarFile to create a tar file,
optionially compressing it with gzip.
Unfortunately go only has gzip compression support, not bzip2 or xz.
However, this is not a regression, as docker currently uses *no*
compression for TarFilter(). The only compression of tarfiles
currently happens in utils/tarsum.go, and that manually does gzip
compression.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This is the code that takes a normal file and adds it to a TarWriter.
We extract it so that we can share it with Tar().
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This simplifies that code that calls out to tar by removing support
for now unused features.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This replaces the shelling out to tar with a reimplementation of untar
based on the archive/tar code and the pre-existing code from ApplyLayer
to create real files from tar headers.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This reverts commit 733bf5d3dd.
This is needed to fix "no such file" errors `docker build` errors for
devicemapper.
This fixes issue #3449.
Docker-DCO-1.0-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
When pulling from a registry we get a compressed tar archive, so
we need to wrap the stream in the right kind of compress reader.
Unfortunately go doesn't have an Xz decompression method, but I
don't think any docker layers use that atm anyway.
