1
0
Fork 0
mirror of https://github.com/moby/moby.git synced 2022-11-09 12:21:53 -05:00
Commit graph

9 commits

Author SHA1 Message Date
Aaron Lehmann
572ce80230 Improved push and pull with upload manager and download manager
This commit adds a transfer manager which deduplicates and schedules
transfers, and also an upload manager and download manager that build on
top of the transfer manager to provide high-level interfaces for uploads
and downloads. The push and pull code is modified to use these building
blocks.

Some benefits of the changes:

- Simplification of push/pull code
- Pushes can upload layers concurrently
- Failed downloads and uploads are retried after backoff delays
- Cancellation is supported, but individual transfers will only be
  cancelled if all pushes or pulls using them are cancelled.
- The distribution code is decoupled from Docker Engine packages and API
  conventions (i.e. streamformatter), which will make it easier to split
  out.

This commit also includes unit tests for the new distribution/xfer
package. The tests cover 87.8% of the statements in the package.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-09 19:13:35 -08:00
Aaron Lehmann
64f8ee444d Cap the amount of buffering done by BytesPipe
Turn BytesPipe's Read and Write functions into blocking, goroutine-safe
functions. Add a CloseWithError function to propagate an error code to
the Read function.

Adjust tests to work with the blocking Read and Write functions.

Remove BufReader, since now its users can use BytesPipe directly.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-11-12 10:11:29 -08:00
Doug Davis
b5265fe7ad Make Close() on simpleReaderCloser actually close the reader
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-24 20:22:59 -07:00
Alexander Morozov
56b70bf84e Refactor bufReader to use BytesPipe
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-09-23 16:00:28 -07:00
root
9ca913d0f1 Go Scheduler issue with sync.Mutex using gccgo
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-07-17 01:33:58 +00:00
vagrant
d2a2c974d9 fix to gccgo compilation error
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-05-14 16:28:23 +00:00
Vincent Demeester
815b472a02 Add more ioutils tests.
Closes #11595

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-05-04 19:56:10 +02:00
unclejack
028f7987fe pkg/ioutils: add tests for BufReader
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-03-18 19:32:46 +02:00
unclejack
76212635b5 move some io related utils to pkg/ioutils
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00