moby--moby/integration/container/run_linux_test.go

294 lines
9.7 KiB
Go
Raw Normal View History

package container // import "github.com/docker/docker/integration/container"
import (
"bytes"
"context"
"io"
"os"
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"github.com/docker/docker/api/types"
containertypes "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/integration/internal/container"
net "github.com/docker/docker/integration/internal/network"
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
"github.com/docker/docker/pkg/stdcopy"
"github.com/docker/docker/pkg/system"
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
"github.com/docker/docker/testutil/daemon"
"golang.org/x/sys/unix"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
"gotest.tools/v3/poll"
"gotest.tools/v3/skip"
)
func TestNISDomainname(t *testing.T) {
// Older versions of the daemon would concatenate hostname and domainname,
// so hostname "foobar" and domainname "baz.cyphar.com" would produce
// `foobar.baz.cyphar.com` as hostname.
skip.If(t, versions.LessThan(testEnv.DaemonAPIVersion(), "1.40"), "skip test from new feature")
skip.If(t, testEnv.DaemonInfo.OSType != "linux")
// Rootless supports custom Hostname but doesn't support custom Domainname
// OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \
// "write sysctl key kernel.domainname: open /proc/sys/kernel/domainname: permission denied\"": unknown.
skip.If(t, testEnv.IsRootless, "rootless mode doesn't support setting Domainname (TODO: https://github.com/moby/moby/issues/40632)")
defer setupTest(t)()
client := testEnv.APIClient()
ctx := context.Background()
const (
hostname = "foobar"
domainname = "baz.cyphar.com"
)
cID := container.Run(ctx, t, client, func(c *container.TestContainerConfig) {
c.Config.Hostname = hostname
c.Config.Domainname = domainname
})
poll.WaitOn(t, container.IsInState(ctx, client, cID, "running"), poll.WithDelay(100*time.Millisecond))
inspect, err := client.ContainerInspect(ctx, cID)
assert.NilError(t, err)
assert.Check(t, is.Equal(hostname, inspect.Config.Hostname))
assert.Check(t, is.Equal(domainname, inspect.Config.Domainname))
// Check hostname.
res, err := container.Exec(ctx, client, cID,
[]string{"cat", "/proc/sys/kernel/hostname"})
assert.NilError(t, err)
assert.Assert(t, is.Len(res.Stderr(), 0))
assert.Equal(t, 0, res.ExitCode)
assert.Check(t, is.Equal(hostname, strings.TrimSpace(res.Stdout())))
// Check domainname.
res, err = container.Exec(ctx, client, cID,
[]string{"cat", "/proc/sys/kernel/domainname"})
assert.NilError(t, err)
assert.Assert(t, is.Len(res.Stderr(), 0))
assert.Equal(t, 0, res.ExitCode)
assert.Check(t, is.Equal(domainname, strings.TrimSpace(res.Stdout())))
}
func TestHostnameDnsResolution(t *testing.T) {
skip.If(t, testEnv.DaemonInfo.OSType != "linux")
defer setupTest(t)()
client := testEnv.APIClient()
ctx := context.Background()
const (
hostname = "foobar"
)
// using user defined network as we want to use internal DNS
netName := "foobar-net"
net.CreateNoError(context.Background(), t, client, netName, net.WithDriver("bridge"))
cID := container.Run(ctx, t, client, func(c *container.TestContainerConfig) {
c.Config.Hostname = hostname
c.HostConfig.NetworkMode = containertypes.NetworkMode(netName)
})
poll.WaitOn(t, container.IsInState(ctx, client, cID, "running"), poll.WithDelay(100*time.Millisecond))
inspect, err := client.ContainerInspect(ctx, cID)
assert.NilError(t, err)
assert.Check(t, is.Equal(hostname, inspect.Config.Hostname))
// Clear hosts file so ping will use DNS for hostname resolution
res, err := container.Exec(ctx, client, cID,
[]string{"sh", "-c", "echo 127.0.0.1 localhost | tee /etc/hosts && ping -c 1 foobar"})
assert.NilError(t, err)
assert.Check(t, is.Equal("", res.Stderr()))
assert.Equal(t, 0, res.ExitCode)
}
func TestUnprivilegedPortsAndPing(t *testing.T) {
skip.If(t, testEnv.DaemonInfo.OSType != "linux")
skip.If(t, testEnv.IsRootless, "rootless mode doesn't support setting net.ipv4.ping_group_range and net.ipv4.ip_unprivileged_port_start")
defer setupTest(t)()
client := testEnv.APIClient()
ctx := context.Background()
cID := container.Run(ctx, t, client, func(c *container.TestContainerConfig) {
c.Config.User = "1000:1000"
})
poll.WaitOn(t, container.IsInState(ctx, client, cID, "running"), poll.WithDelay(100*time.Millisecond))
// Check net.ipv4.ping_group_range.
res, err := container.Exec(ctx, client, cID, []string{"cat", "/proc/sys/net/ipv4/ping_group_range"})
assert.NilError(t, err)
assert.Assert(t, is.Len(res.Stderr(), 0))
assert.Equal(t, 0, res.ExitCode)
assert.Equal(t, `0 2147483647`, strings.TrimSpace(res.Stdout()))
// Check net.ipv4.ip_unprivileged_port_start.
res, err = container.Exec(ctx, client, cID, []string{"cat", "/proc/sys/net/ipv4/ip_unprivileged_port_start"})
assert.NilError(t, err)
assert.Assert(t, is.Len(res.Stderr(), 0))
assert.Equal(t, 0, res.ExitCode)
assert.Equal(t, "0", strings.TrimSpace(res.Stdout()))
}
func TestPrivilegedHostDevices(t *testing.T) {
// Host devices are linux only. Also it creates host devices,
// so needs to be same host.
skip.If(t, testEnv.IsRemoteDaemon)
skip.If(t, testEnv.DaemonInfo.OSType != "linux")
defer setupTest(t)()
client := testEnv.APIClient()
ctx := context.Background()
const (
devTest = "/dev/test"
devRootOnlyTest = "/dev/root-only/test"
)
// Create Null devices.
if err := system.Mknod(devTest, unix.S_IFCHR|0600, int(system.Mkdev(1, 3))); err != nil {
t.Fatal(err)
}
defer os.Remove(devTest)
if err := os.Mkdir(filepath.Dir(devRootOnlyTest), 0700); err != nil {
t.Fatal(err)
}
defer os.RemoveAll(filepath.Dir(devRootOnlyTest))
if err := system.Mknod(devRootOnlyTest, unix.S_IFCHR|0600, int(system.Mkdev(1, 3))); err != nil {
t.Fatal(err)
}
defer os.Remove(devRootOnlyTest)
cID := container.Run(ctx, t, client, container.WithPrivileged(true))
poll.WaitOn(t, container.IsInState(ctx, client, cID, "running"), poll.WithDelay(100*time.Millisecond))
// Check test device.
res, err := container.Exec(ctx, client, cID, []string{"ls", devTest})
assert.NilError(t, err)
assert.Equal(t, 0, res.ExitCode)
assert.Check(t, is.Equal(strings.TrimSpace(res.Stdout()), devTest))
// Check root-only test device.
res, err = container.Exec(ctx, client, cID, []string{"ls", devRootOnlyTest})
assert.NilError(t, err)
if testEnv.IsRootless() {
assert.Equal(t, 1, res.ExitCode)
assert.Check(t, is.Contains(res.Stderr(), "No such file or directory"))
} else {
assert.Equal(t, 0, res.ExitCode)
assert.Check(t, is.Equal(strings.TrimSpace(res.Stdout()), devRootOnlyTest))
}
}
func TestRunConsoleSize(t *testing.T) {
skip.If(t, testEnv.DaemonInfo.OSType != "linux")
skip.If(t, versions.LessThan(testEnv.DaemonAPIVersion(), "1.42"), "skip test from new feature")
defer setupTest(t)()
client := testEnv.APIClient()
ctx := context.Background()
cID := container.Run(ctx, t, client,
container.WithTty(true),
container.WithImage("busybox"),
container.WithCmd("stty", "size"),
container.WithConsoleSize(57, 123),
)
poll.WaitOn(t, container.IsStopped(ctx, client, cID), poll.WithDelay(100*time.Millisecond))
out, err := client.ContainerLogs(ctx, cID, types.ContainerLogsOptions{ShowStdout: true})
assert.NilError(t, err)
defer out.Close()
var b bytes.Buffer
_, err = io.Copy(&b, out)
assert.NilError(t, err)
assert.Equal(t, strings.TrimSpace(b.String()), "123 57")
}
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
func TestRunWithAlternativeContainerdShim(t *testing.T) {
skip.If(t, testEnv.IsRemoteDaemon)
skip.If(t, testEnv.DaemonInfo.OSType != "linux")
realShimPath, err := exec.LookPath("containerd-shim-runc-v2")
assert.Assert(t, err)
realShimPath, err = filepath.Abs(realShimPath)
assert.Assert(t, err)
// t.TempDir() can't be used here as the temporary directory returned by
// that function cannot be accessed by the fake-root user for rootless
// Docker. It creates a nested hierarchy of directories where the
// outermost has permission 0700.
shimDir, err := os.MkdirTemp("", t.Name())
assert.Assert(t, err)
t.Cleanup(func() {
if err := os.RemoveAll(shimDir); err != nil {
t.Errorf("shimDir RemoveAll cleanup: %v", err)
}
})
assert.Assert(t, os.Chmod(shimDir, 0777))
shimDir, err = filepath.Abs(shimDir)
assert.Assert(t, err)
assert.Assert(t, os.Symlink(realShimPath, filepath.Join(shimDir, "containerd-shim-realfake-v42")))
d := daemon.New(t,
daemon.WithEnvVars("PATH="+shimDir+":"+os.Getenv("PATH")),
daemon.WithContainerdSocket(""), // A new containerd instance needs to be started which inherits the PATH env var defined above.
)
d.StartWithBusybox(t)
defer d.Stop(t)
client := d.NewClientT(t)
ctx := context.Background()
cID := container.Run(ctx, t, client,
container.WithImage("busybox"),
container.WithCmd("sh", "-c", `echo 'Hello, world!'`),
container.WithRuntime("io.containerd.realfake.v42"),
)
poll.WaitOn(t, container.IsStopped(ctx, client, cID), poll.WithDelay(100*time.Millisecond))
out, err := client.ContainerLogs(ctx, cID, types.ContainerLogsOptions{ShowStdout: true})
assert.NilError(t, err)
defer out.Close()
var b bytes.Buffer
_, err = stdcopy.StdCopy(&b, io.Discard, out)
assert.NilError(t, err)
assert.Equal(t, strings.TrimSpace(b.String()), "Hello, world!")
d.Stop(t)
d.Start(t, "--default-runtime="+"io.containerd.realfake.v42")
cID = container.Run(ctx, t, client,
container.WithImage("busybox"),
container.WithCmd("sh", "-c", `echo 'Hello, world!'`),
)
poll.WaitOn(t, container.IsStopped(ctx, client, cID), poll.WithDelay(100*time.Millisecond))
out, err = client.ContainerLogs(ctx, cID, types.ContainerLogsOptions{ShowStdout: true})
assert.NilError(t, err)
defer out.Close()
b.Reset()
_, err = stdcopy.StdCopy(&b, io.Discard, out)
assert.NilError(t, err)
assert.Equal(t, strings.TrimSpace(b.String()), "Hello, world!")
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
}