From 4a74a46f4485552dd4e9c7ffa52c8428954cb507 Mon Sep 17 00:00:00 2001 From: Brian Goff Date: Sun, 28 Oct 2018 16:54:23 -0700 Subject: [PATCH] Update roadmap to reflect reality. The roadmap is one of the most important ways that a new contributor may get started on the codebase, as such it is important for it to reflect the real effort that is currently happening. This update just brings it up to date. There may be some other efforts going on and I would encourage people to update the roadmap accordingly as a separate effort. Signed-off-by: Brian Goff --- ROADMAP.md | 91 +++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 70 insertions(+), 21 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index e2e6b2b960..05e33b4bcd 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -35,34 +35,83 @@ issue, in the Slack channel, or in person at the Moby Summits that happen every ## 1.1 Runtime improvements -We introduced [`runC`](https://runc.io) as a standalone low-level tool for container -execution in 2015, the first stage in spinning out parts of the Engine into standalone tools. +Over time we have accumulated a lot of functionality in the container runtime +aspect of Moby while also growing in other areas. Much of the container runtime +pieces are now duplicated work available in other, lower level components such +as [containerd](https://containerd.io). -As runC continued evolving, and the OCI specification along with it, we created -[`containerd`](https://github.com/containerd/containerd), a daemon to control and monitor `runC`. -In late 2016 this was relaunched as the `containerd` 1.0 track, aiming to provide a common runtime -for the whole spectrum of container systems, including Kubernetes, with wide community support. -This change meant that there was an increased scope for `containerd`, including image management -and storage drivers. +Moby currently only utilizes containerd for basic runtime state management, e.g. starting +and stopping a container, which is what the pre-containerd 1.0 daemon provided. +Now that containerd is a full-fledged container runtime which supports full +container life-cycle management, we would like to start relying more on containerd +and removing the bits in Moby which are now duplicated. This will neccessitate +a signficant effort to refactor and even remove large parts of Moby's codebase. -Moby will rely on a long-running `containerd` companion daemon for all container execution -related operations. This could open the door in the future for Engine restarts without interrupting -running containers. The switch over to containerd 1.0 is an important goal for the project, and -will result in a significant simplification of the functions implemented in this repository. +Tracking issues: -## 1.2 Internal decoupling +- [#38043](https://github.com/moby/moby/issues/38043) Proposal: containerd image integration + +## 1.2 Image Builder + +Work is ongoing to integrate [BuildKit](https://github.com/moby/buildkit) into +Moby and replace the "v0" build implementation. Buildkit offers better cache +management, parallelizable build steps, and better extensibility while also +keeping builds portable, a chief tenent of Moby's builder. + +Upon completion of this effort, users will have a builder that performs better +while also being more extensible, enabling users to provide their own custom +syntax which can be either Dockerfile-like or something completely different. + +See [buildpacks on buildkit](https://github.com/tonistiigi/buildkit-pack) as an +example of this extensibility. + +New features for the builder and Dockerfile should be implemented first in the +BuildKit backend using an external Dockerfile implementation from the container +images. This allows everyone to test and evaluate the feature without upgrading +their daemon. New features should go to the experimental channel first, and can be +part of the `docker/dockerfile:experimental` image. From there they graduate to +`docker/dockerfile:latest` and binary releases. The Dockerfile frontend source +code is temporarily located at +[https://github.com/moby/buildkit/tree/master/frontend/dockerfile](https://github.com/moby/buildkit/tree/master/frontend/dockerfile) +with separate new features defined with go build tags. + +Tracking issues: + +- [#32925](https://github.com/moby/moby/issues/32925) discussion: builder future: buildkit + +## 1.3 Rootless Mode + +Running the daemon requires elevated privileges for many tasks. We would like to +support running the daemon as a nomral, unprivileged user without requiring `suid` +binaries. + +Tracking issues: + +- [#37375](https://github.com/moby/moby/issues/37375) Proposal: allow running `dockerd` as an unprivileged user (aka rootless mode) + +## 1.4 Testing + +Moby has many tests, both unit and integration. Moby needs more tests which can +cover the full spectrum functionality and edge cases out there. + +Tests in the `integration-cli` folder should also be migrated into (both in +location and style) the `integration` folder. These newer tests are simpler to +run in isolation, simpler to read, simpler to write, and more fully exercise the +API. Meanwhile tests of the docker CLI should generally live in docker/cli. + +Tracking issues: + +- [#32866](https://github.com/moby/moby/issues/32866) Replace integration-cli suite with API test suite + +## 1.5 Internal decoupling A lot of work has been done in trying to decouple Moby internals. This process of creating standalone projects with a well defined function that attract a dedicated community should continue. As well as integrating `containerd` we would like to integrate [BuildKit](https://github.com/moby/buildkit) as the next standalone component. - We see gRPC as the natural communication layer between decoupled components. -## 1.3 Custom assembly tooling - -We have been prototyping the Moby [assembly tool](https://github.com/moby/tool) which was originally -developed for LinuxKit and intend to turn it into a more generic packaging and assembly mechanism -that can build not only the default version of Moby, as distribution packages or other useful forms, -but can also build very different container systems, themselves built of cooperating daemons built in -and running in containers. We intend to merge this functionality into this repo. +In addition to pushing out large components into other projects, much of the +internal code structure, and in particular the +["Daemon"](https://godoc.org/github.com/docker/docker/daemon#Daemon) object, +should be split into smaller, more manageable, and more testable components.