This seems clearer to me and avoids a miniscule amount of string formatting.
I have deliberately not changed the tests here to avoid the possibility of
changing both wrongly.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Although our use of ANSI codes here is rather simple it is generally good
practice to use terminfo in order to be portable to different terminal
emulators.
Vendor github.com/Nvveen/Gotty (actually my fork with a fix, see
https://github.com/Nvveen/Gotty/pull/1) and use that to parse the terminfo
files.
Note that "\e]2K" (clear entire line) is not covered by terminfo. We can
achieve the same end by first clearing from begining of line to cursor
(el1="\e]1K") and then clearing from cursor to end of line (el="\e]k").
Test suite has been updated and forces (either directly or by setting $TERM to
something highly unlikely to exist) the use of the non-terminfo fallbacks which
retains the same output behaviour as previously. This is preferable even to
relying on a well-known and relatively static terminfo (like vt102) since even
that in principal might have different terminfo encodings.
In case terminfo is not available at all for $TERM or doesn't expose the
specific capabilities which we use then fall back to the previous manual
escapes, with the exception that we avoid "\e]2K" as discussed above.
Tested with a manual docker pull with rxvt-unicode ($TERM=rxvt-unicode), xterm
($TERM=xterm), mlterm ($TERM=mlterm) and aterm ($TERM=kterm).
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
When logging the failure of a case which involves control characters add line
feeds and spacing such that the expected and actual strings are aligned in the
same columns making comparisons easier.
Turns:
jsonmessage_test.go:156: Expected "\x1b[2K\rstatus 1 B\r", got "\x1b[1K\x1b[K\rstatus 1 B\r"
Into:
jsonmessage_test.go:156:
Expected "\x1b[2K\rstatus 1 B\r"
got "\x1b[1K\x1b[K\rstatus 1 B\r"
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
The majority already did so, but a few used %v.
Compared with %v using %q escapes any control characters ensuring they are
visible in the logs (otherwise they can cause scrolling and overwriting etc).
The %q formatter also includes the surrounding "s making the manual use of []
to wrap/delimit the %v output unnecessary.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
The ANSI escape codes \e[0A (cursor up 0 lines) and \e[0B (cursor down 0 lines)
are not well defined and are treated differently by different terminals. In
particular xterm treats 0 as a missing parameter and therefore defaults to 1,
whereas rxvt-unicode treats these escapes as a request to move 0 lines.
However the use of these codes is unnecessary and were really just hiding the
fact that we were not correctly computing diff when adding a new line. Having
added the new line to the ids map and output the corresponding \n we need to
then calculate a correct diff of 1 rather than leaving it as the default 0
(which xterm then interprets as 1). The fix is to pull the diff calculation out
of the else case and to always do it.
With this in place we can then avoid outputting escapes for moving 0 lines.
Actually diff should never be 0 to start with any more, but check to be safe.
This fixes corruption of `docker pull` seen with rxvt-unicode (and likely other
terminals in that family) seen in #28111. Tested with rxvt-unicode
($TERM=rxvt-unicode), xterm ($TERM=xterm), mlterm ($TERM=mlterm) and aterm
($TERM=kterm).
The test cases have been updated to match the new behaviour.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
The trust code used to parse the console output of `docker push` to
extract the digest, tag, and size information and determine what to
sign. This is fragile and might give an attacker control over what gets
signed if the attacker can find a way to influence what gets printed as
part of the push output.
This commit sends the push metadata out-of-band. It introduces an `Aux`
field in JSONMessage that can carry application-specific data alongside
progress updates. Instead of parsing formatted output, the client looks
in this field to get the digest, size, and tag from the push.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
- Move time json marshaling to the jsonlog package: this is a docker
internal hack that we should not promote as a library.
- Move Timestamp encoding/decoding functions to the API types: This is
only used there. It could be a standalone library but I don't this
it's worth having a separated repo for this. It could introduce more
complexity than it solves.
Signed-off-by: David Calavera <david.calavera@gmail.com>
In the existing code, "diff" has function scope and the value from the
previous iteration may be used if it is not reset. This appears to be an
oversight. This commit changes its scope to the for loop body.
One confusing point is that the cursor movement escape sequences appear
to be necessary even if the requested movement is 0. I haven't been able
to figure out why this makes a difference.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
When we handle a message that isn't tracked in the "line" map (for
example, one with no ID), clear the line map. This means we won't update
lines that were part of a previous, completed set of operations when
doing something like pull -a. It also has the beneficial side effect
of avoiding terminal glitching in these types of situations, since
messages that don't get tracked in the "line" map cause the count of the
number of lines to get out of sync.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Although having a request ID available throughout the codebase is very
valuable, the impact of requiring a Context as an argument to every
function in the codepath of an API request, is too significant and was
not properly understood at the time of the review.
Furthermore, mixing API-layer code with non-API-layer code makes the
latter usable only by API-layer code (one that has a notion of Context).
This reverts commit de41640435, reversing
changes made to 7daeecd42d.
Signed-off-by: Tibor Vass <tibor@docker.com>
Conflicts:
api/server/container.go
builder/internals.go
daemon/container_unix.go
daemon/create.go
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:
```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.
Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.
I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.
ping @icecrime @calavera @crosbymichael
Signed-off-by: Doug Davis <dug@us.ibm.com>
This way provide both Time and TimeNano in the event. For the display of
the JSONMessage, use either, but prefer TimeNano Proving only TimeNano
would break Subscribers that are using the `Time` field, so both are set
for backwards compatibility.
The events logging uses nano formatting, but only provides a Unix()
time, therefor ordering may get lost in the output. Example:
```
2015-09-15T14:18:51.000000000-04:00 ee46febd64ac629f7de9cd8bf58582e6f263d97ff46896adc5b508db804682da: (from busybox) resize
2015-09-15T14:18:51.000000000-04:00 a78c9149b1c0474502a117efaa814541926c2ae6ec3c76607e1c931b84c3a44b: (from busybox) resize
```
By having a field just for Nano time, when set, the marshalling back to
`time.Unix(sec int64, nsec int64)` has zeros exactly where it needs to.
This does not break any existing use of jsonmessage.JSONMessage, but now
allows for use of `UnixNano()` and get event formatting that has
distinguishable order. Example:
```
2015-09-15T15:37:23.810295632-04:00 6adcf8ed9f5f5ec059a915466cd1cde86a18b4a085fc3af405e9cc9fecbbbbaf: (from busybox) resize
2015-09-15T15:37:23.810412202-04:00 6b7c5bfdc3f902096f5a91e628f21bd4b56e32590c5b4b97044aafc005ddcb0d: (from busybox) resize
```
Including tests for TimeNano and updated event API reference doc.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
The practice of buffering to a tempfile during a pushing contributes massively
to slow V2 push performance perception. The protocol was actually designed to
avoid precalculation, supporting cut-through data push. This means we can
assemble the layer, calculate its digest and push to the remote endpoint, all
at the same time.
This should increase performance massively on systems with slow disks or IO
bottlenecks.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Some structures use int for sizes and UNIX timestamps. On some
platforms, int is 32 bits, so this can lead to the year 2038 issues and
overflows when dealing with large containers or layers.
Consistently use int64 to store sizes and UNIX timestamps in
api/types/types.go. Update related to code accordingly (i.e.
strconv.FormatInt instead of strconv.Itoa).
Use int64 in progressreader package to avoid integer overflow when
dealing with large quantities. Update related code accordingly.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
