1
0
Fork 0
mirror of https://github.com/moby/moby.git synced 2022-11-09 12:21:53 -05:00
Commit graph

17 commits

Author SHA1 Message Date
Aaron Lehmann
572ce80230 Improved push and pull with upload manager and download manager
This commit adds a transfer manager which deduplicates and schedules
transfers, and also an upload manager and download manager that build on
top of the transfer manager to provide high-level interfaces for uploads
and downloads. The push and pull code is modified to use these building
blocks.

Some benefits of the changes:

- Simplification of push/pull code
- Pushes can upload layers concurrently
- Failed downloads and uploads are retried after backoff delays
- Cancellation is supported, but individual transfers will only be
  cancelled if all pushes or pulls using them are cancelled.
- The distribution code is decoupled from Docker Engine packages and API
  conventions (i.e. streamformatter), which will make it easier to split
  out.

This commit also includes unit tests for the new distribution/xfer
package. The tests cover 87.8% of the statements in the package.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-09 19:13:35 -08:00
Aaron Lehmann
64f8ee444d Cap the amount of buffering done by BytesPipe
Turn BytesPipe's Read and Write functions into blocking, goroutine-safe
functions. Add a CloseWithError function to propagate an error code to
the Read function.

Adjust tests to work with the blocking Read and Write functions.

Remove BufReader, since now its users can use BytesPipe directly.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-11-12 10:11:29 -08:00
Srini Brahmaroutu
7c52fcce85 goroutine fairness is not guaranteed causing the hang with GCCGO
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-09-25 14:40:16 +00:00
Tonis Tiigi
c5b23337c3 Make bytesPipe use linear allocations
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-09-23 17:12:54 -07:00
Alexander Morozov
56b70bf84e Refactor bufReader to use BytesPipe
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-09-23 16:00:28 -07:00
unclejack
f9df813a99 pkg/ioutils: lower bufReader reset timeout
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-08-28 23:13:02 +03:00
Veres Lajos
5146232723 typofix - https://github.com/vlajos/misspell_fixer
Signed-off-by: Veres Lajos <vlajos@gmail.com>
2015-08-07 23:25:49 +01:00
Lei Jitang
27a8f9937e Enable golint part of #14756
pkg/broadcastwriter
pkg/graphdb
pkg/httputils
pkg/ioutils

Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-08-03 09:45:05 +08:00
David Calavera
40ea67a596 Fix reset timeout for buffer readers.
Use our goroutine-safe random source.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-07-28 14:30:18 -07:00
David Calavera
7b83b0e15c Merge pull request #14605 from brahmaroutu/gccgo_scheduler
Go Scheduler issue with sync.Mutex
2015-07-17 08:16:32 -07:00
root
9ca913d0f1 Go Scheduler issue with sync.Mutex using gccgo
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-07-17 01:33:58 +00:00
Tibor Vass
276c640be4 remove pkg/transport and use the one from distribution
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-07-16 13:13:46 -04:00
Antonio Murdaca
c30a55f14d Refactor utils/utils, fixes #11923
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-04-14 01:37:36 +02:00
unclejack
e5ea2b2357 pkg/ioutils: avoid huge Buffer growth in bufreader
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-03-18 19:51:55 +02:00
unclejack
bd130e72a0 pkg/ioutils: add ReaderErrWrapper to readers
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00
unclejack
c93b9e81c0 pkg/ioutils: add NewBufReaderWithDrainbufAndBuffer
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00
unclejack
76212635b5 move some io related utils to pkg/ioutils
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00