Moby Project - a collaborative project for the container ecosystem to assemble container-based systems
Go to file
John Howard 20833b06a0 Windows: (WCOW) Generate OCI spec that remote runtime can escape
Signed-off-by: John Howard <jhoward@microsoft.com>

Also fixes https://github.com/moby/moby/issues/22874

This commit is a pre-requisite to moving moby/moby on Windows to using
Containerd for its runtime.

The reason for this is that the interface between moby and containerd
for the runtime is an OCI spec which must be unambigious.

It is the responsibility of the runtime (runhcs in the case of
containerd on Windows) to ensure that arguments are escaped prior
to calling into HCS and onwards to the Win32 CreateProcess call.

Previously, the builder was always escaping arguments which has
led to several bugs in moby. Because the local runtime in
libcontainerd had context of whether or not arguments were escaped,
it was possible to hack around in daemon/oci_windows.go with
knowledge of the context of the call (from builder or not).

With a remote runtime, this is not possible as there's rightly
no context of the caller passed across in the OCI spec. Put another
way, as I put above, the OCI spec must be unambigious.

The other previous limitation (which leads to various subtle bugs)
is that moby is coded entirely from a Linux-centric point of view.

Unfortunately, Windows != Linux. Windows CreateProcess uses a
command line, not an array of arguments. And it has very specific
rules about how to escape a command line. Some interesting reading
links about this are:

https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/
https://stackoverflow.com/questions/31838469/how-do-i-convert-argv-to-lpcommandline-parameter-of-createprocess
https://docs.microsoft.com/en-us/cpp/cpp/parsing-cpp-command-line-arguments?view=vs-2017

For this reason, the OCI spec has recently been updated to cater
for more natural syntax by including a CommandLine option in
Process.

What does this commit do?

Primary objective is to ensure that the built OCI spec is unambigious.

It changes the builder so that `ArgsEscaped` as commited in a
layer is only controlled by the use of CMD or ENTRYPOINT.

Subsequently, when calling in to create a container from the builder,
if follows a different path to both `docker run` and `docker create`
using the added `ContainerCreateIgnoreImagesArgsEscaped`. This allows
a RUN from the builder to control how to escape in the OCI spec.

It changes the builder so that when shell form is used for RUN,
CMD or ENTRYPOINT, it builds (for WCOW) a more natural command line
using the original as put by the user in the dockerfile, not
the parsed version as a set of args which loses fidelity.
This command line is put into args[0] and `ArgsEscaped` is set
to true for CMD or ENTRYPOINT. A RUN statement does not commit
`ArgsEscaped` to the commited layer regardless or whether shell
or exec form were used.
2019-03-12 18:41:55 -07:00
.github Remove myself from codeowners 😅 2019-01-10 17:32:26 +01:00
api Windows: (WCOW) Generate OCI spec that remote runtime can escape 2019-03-12 18:41:55 -07:00
builder Windows: (WCOW) Generate OCI spec that remote runtime can escape 2019-03-12 18:41:55 -07:00
cli allow running `dockerd` in an unprivileged user namespace (rootless mode) 2019-02-04 00:24:27 +09:00
client Add HEAD support for /_ping endpoint 2019-01-31 18:18:24 +01:00
cmd/dockerd Windows: Experimental: Allow containerd for runtime 2019-03-12 18:41:55 -07:00
container Add pids-limit support in docker update 2019-02-21 14:17:38 -08:00
contrib allow running `dockerd` in an unprivileged user namespace (rootless mode) 2019-02-04 00:24:27 +09:00
daemon Windows: (WCOW) Generate OCI spec that remote runtime can escape 2019-03-12 18:41:55 -07:00
distribution Update containerd client to 1.2.4 2019-02-14 04:47:27 +01:00
dockerversion Remove version-checks for containerd and runc 2018-10-04 23:17:13 +02:00
docs Add new PidsLimit options to API version history 2019-02-24 14:27:30 +01:00
errdefs Make errdefs helpers idempotent 2019-01-03 11:16:01 +01:00
hack fix hack/dockerfile/install/containerd.installer test statement 2019-02-26 18:19:04 +08:00
image Windows: (WCOW) Generate OCI spec that remote runtime can escape 2019-03-12 18:41:55 -07:00
integration Merge pull request #38428 from thaJeztah/only_create_new_daemon_if_needed 2019-02-25 22:20:05 -08:00
integration-cli Windows: (WCOW) Generate OCI spec that remote runtime can escape 2019-03-12 18:41:55 -07:00
internal Testing: create new daemon (only) if needed 2019-02-23 13:32:59 +01:00
layer layer/layer_store: ensure NewInputTarStream resources are released 2018-12-21 09:30:09 +01:00
libcontainerd Windows: (WCOW) Generate OCI spec that remote runtime can escape 2019-03-12 18:41:55 -07:00
oci Capabilities refactor 2019-01-22 21:50:41 +02:00
opts allow running `dockerd` in an unprivileged user namespace (rootless mode) 2019-02-04 00:24:27 +09:00
pkg Windows: (WCOW) Generate OCI spec that remote runtime can escape 2019-03-12 18:41:55 -07:00
plugin Windows: Experimental: Allow containerd for runtime 2019-03-12 18:41:55 -07:00
profiles seccomp: review update 2019-02-05 12:02:41 -08:00
project REVIEWING.md: Fix status 4 merge label 2019-01-23 02:23:30 +01:00
reference Merge pull request #37781 from mtrmac/reference-race-upstream 2018-10-18 12:35:57 -07:00
registry registry: use len(via)!=0 instead of via!=nil 2018-12-11 16:37:16 +03:00
reports
restartmanager
rootless allow running `dockerd` in an unprivileged user namespace (rootless mode) 2019-02-04 00:24:27 +09:00
runconfig Format code with gofmt -s from go-1.11beta1 2018-09-06 15:24:16 -07:00
vendor Vendor sirupsen/logrus@v1.3.0 2019-03-12 18:41:55 -07:00
volume Use assert.NilError() instead of assert.Assert() 2019-01-21 13:16:02 +01:00
.DEREK.yml .DEREK.yml: add myself 2019-01-10 16:40:29 -08:00
.dockerignore
.gitignore
.mailmap Update authors and mailmap 2018-09-07 23:43:34 +08:00
AUTHORS Update authors and mailmap 2018-09-07 23:43:34 +08:00
CHANGELOG.md Fix some typos 2018-09-07 13:13:47 +08:00
CONTRIBUTING.md
Dockerfile hack: restore bundling vpnkit on amd64 2019-02-05 18:21:30 -08:00
Dockerfile.e2e Bump Golang 1.11.5 (CVE-2019-6486) 2019-01-24 00:49:27 +01:00
Dockerfile.simple Bump Golang 1.11.5 (CVE-2019-6486) 2019-01-24 00:49:27 +01:00
Dockerfile.windows Bump Golang 1.11.5 (CVE-2019-6486) 2019-01-24 00:49:27 +01:00
LICENSE Update LICENSE 2018-09-12 14:27:53 +01:00
MAINTAINERS Remove "docs maintainers" section 2019-01-23 16:58:58 +01:00
Makefile hack: no need to git fetch in CI 2019-02-05 02:54:50 +00:00
NOTICE
README.md
ROADMAP.md Fix some typos in ROADMAP.md 2019-01-25 14:27:13 +08:00
TESTING.md Merge pull request #37249 from AntaresS/add-test-guidline 2018-06-25 20:44:42 +02:00
VENDORING.md
codecov.yml
poule.yml Poule:Add Windows RS5 2018-10-08 15:38:27 -07:00
vendor.conf Vendor sirupsen/logrus@v1.3.0 2019-03-12 18:41:55 -07:00

README.md

The Moby Project

Moby Project logo

Moby is an open-source project created by Docker to enable and accelerate software containerization.

It provides a "Lego set" of toolkit components, the framework for assembling them into custom container-based systems, and a place for all container enthusiasts and professionals to experiment and exchange ideas. Components include container build tools, a container registry, orchestration tools, a runtime and more, and these can be used as building blocks in conjunction with other tools and projects.

Principles

Moby is an open project guided by strong principles, aiming to be modular, flexible and without too strong an opinion on user experience. It is open to the community to help set its direction.

  • Modular: the project includes lots of components that have well-defined functions and APIs that work together.
  • Batteries included but swappable: Moby includes enough components to build fully featured container system, but its modular architecture ensures that most of the components can be swapped by different implementations.
  • Usable security: Moby provides secure defaults without compromising usability.
  • Developer focused: The APIs are intended to be functional and useful to build powerful tools. They are not necessarily intended as end user tools but as components aimed at developers. Documentation and UX is aimed at developers not end users.

Audience

The Moby Project is intended for engineers, integrators and enthusiasts looking to modify, hack, fix, experiment, invent and build systems based on containers. It is not for people looking for a commercially supported system, but for people who want to work and learn with open source code.

Relationship with Docker

The components and tools in the Moby Project are initially the open source components that Docker and the community have built for the Docker Project. New projects can be added if they fit with the community goals. Docker is committed to using Moby as the upstream for the Docker Product. However, other projects are also encouraged to use Moby as an upstream, and to reuse the components in diverse ways, and all these uses will be treated in the same way. External maintainers and contributors are welcomed.

The Moby project is not intended as a location for support or feature requests for Docker products, but as a place for contributors to work on open source code, fix bugs, and make the code more useful. The releases are supported by the maintainers, community and users, on a best efforts basis only, and are not intended for customers who want enterprise or commercial support; Docker EE is the appropriate product for these use cases.


Legal

Brought to you courtesy of our legal counsel. For more context, please see the NOTICE document in this repo.

Use and transfer of Moby may be subject to certain restrictions by the United States and other governments.

It is your responsibility to ensure that your use and/or transfer does not violate applicable laws.

For more information, please see https://www.bis.doc.gov

Licensing

Moby is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.