kotovalexarian-likes-github/moby--moby
1
0
Fork 0
mirror of https://github.com/moby/moby.git synced 2022-11-09 12:21:53 -05:00
Code Releases Activity

Merged master branch into fs

Browse source
This commit is contained in:
shin- 2013-03-11 05:42:36 -07:00
commit 97a8209438
30 changed files with 1507 additions and 453 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
.gitignore vendored
View file

@ -1,4 +1,5 @@
docker .vagrant
dockerd docker/docker
dockerd/dockerd
.*.swp .*.swp
a.out a.out

184
README.md
View file

@ -23,7 +23,7 @@ Notable features
* Resource isolation: system resources like cpu and memory can be allocated differently to each process container, using cgroups. * Resource isolation: system resources like cpu and memory can be allocated differently to each process container, using cgroups.
* Network isolation: each process container runs in its own network namespace, with a virtual interface and IP address of its own (COMING SOON) * Network isolation: each process container runs in its own network namespace, with a virtual interface and IP address of its own.
* Copy-on-write: root filesystems are created using copy-on-write, which makes deployment extremeley fast, memory-cheap and disk-cheap. * Copy-on-write: root filesystems are created using copy-on-write, which makes deployment extremeley fast, memory-cheap and disk-cheap.
@ -34,6 +34,56 @@ Notable features
* Interactive shell: docker can allocate a pseudo-tty and attach to the standard input of any container, for example to run a throaway interactive shell. * Interactive shell: docker can allocate a pseudo-tty and attach to the standard input of any container, for example to run a throaway interactive shell.
Under the hood
--------------
Under the hood, Docker is built on the following components:
* The [cgroup](http://blog.dotcloud.com/kernel-secrets-from-the-paas-garage-part-24-c) and [namespacing](http://blog.dotcloud.com/under-the-hood-linux-kernels-on-dotcloud-part) capabilities of the Linux kernel;
* [AUFS](http://aufs.sourceforge.net/aufs.html), a powerful union filesystem with copy-on-write capabilities;
* The [Go](http://golang.org) programming language;
* [lxc](http://lxc.sourceforge.net/), a set of convenience scripts to simplify the creation of linux containers.
Setup instructions
==================
Requirements
------------
Right now, the officially supported distributions are:
* Ubuntu 12.04 (precise LTS)
* Ubuntu 12.10 (quantal)
Docker probably works on other distributions featuring a recent kernel, the AUFS patch, and up-to-date lxc. However this has not been tested.
Installation
---------------
1. Set up your host of choice on a physical / virtual machine
2. Assume root identity on your newly installed environment (`sudo -s`)
3. Type the following commands:
apt-get update
apt-get install lxc wget bsdtar curl
4. Download the latest docker binaries: `wget http://docker.io.s3.amazonaws.com/builds/$(uname -s)/$(uname -m)/docker-master.tgz` ([Or get the Linux/x86_64 binaries here](http://docker.io.s3.amazonaws.com/builds/Linux/x86_64/docker-master.tgz) )
5. Extract the contents of the tar file `tar -xf docker-master.tar.gz`
6. Launch the docker daemon in the background `./dockerd &`
7. Download a base image `./docker pull base`
8. Run your first container! `./docker run -i -a -t base /bin/bash`
9. Start exploring `./docker --help`
Consider adding docker and dockerd to your `PATH` for simplicity.
What is a Standard Container? What is a Standard Container?
----------------------------- -----------------------------
@ -76,20 +126,6 @@ With Standard Containers we can put an end to that embarrassment, by making INDU
Under the hood
--------------
Under the hood, Docker is built on the following components:
* The [cgroup](http://blog.dotcloud.com/kernel-secrets-from-the-paas-garage-part-24-c) and [namespacing](http://blog.dotcloud.com/under-the-hood-linux-kernels-on-dotcloud-part) capabilities of the Linux kernel;
* [AUFS](http://aufs.sourceforge.net/aufs.html), a powerful union filesystem with copy-on-write capabilities;
* The [Go](http://golang.org) programming language;
* [lxc](http://lxc.sourceforge.net/), a set of convenience scripts to simplify the creation of linux containers.
Standard Container Specification Standard Container Specification
-------------------------------- --------------------------------
@ -135,121 +171,3 @@ Standard Container Specification
#### Security #### Security
Setup instructions
==================
Requirements
------------
Right now, the officially supported distributions are:
* Ubuntu 12.04 (precise LTS)
* Ubuntu 12.10 (quantal)
Docker probably works on other distributions featuring a recent kernel, the AUFS patch, and up-to-date lxc. However this has not been tested.
Step by step host setup
-----------------------
1. Set up your host of choice on a physical / virtual machine
2. Assume root identity on your newly installed environment (`sudo -s`)
3. Type the following commands:
apt-get update
apt-get install lxc wget
4. Download the latest version of the [docker binaries](https://dl.dropbox.com/u/20637798/docker.tar.gz) (`wget https://dl.dropbox.com/u/20637798/docker.tar.gz`) (warning: this may not be the most up-to-date build)
5. Extract the contents of the tar file `tar -xf docker.tar.gz`
6. Launch the docker daemon `./dockerd`
7. Download a base image by running 'docker pull -j base'
Client installation
-------------------
4. Download the latest version of the [docker binaries](https://dl.dropbox.com/u/20637798/docker.tar.gz) (`wget https://dl.dropbox.com/u/20637798/docker.tar.gz`)
5. Extract the contents of the tar file `tar -xf docker.tar.gz`
6. You can now use the docker client binary `./docker`. Consider adding it to your `PATH` for simplicity.
Vagrant Usage
-------------
1. Install Vagrant from http://vagrantup.com
2. Run `vagrant up`. This will take a few minutes as it does the following:
- Download Quantal64 base box
- Kick off Puppet to do:
- Download & untar most recent docker binary tarball to vagrant homedir.
- Debootstrap to /var/lib/docker/images/ubuntu.
- Install & run dockerd as service.
- Put docker in /usr/local/bin.
- Put latest Go toolchain in /usr/local/go.
Sample run output:
```bash
$ vagrant up
[default] Importing base box 'quantal64'...
[default] Matching MAC address for NAT networking...
[default] Clearing any previously set forwarded ports...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.
[default] VM booted and ready for use!
[default] Mounting shared folders...
[default] -- v-root: /vagrant
[default] -- manifests: /tmp/vagrant-puppet/manifests
[default] -- v-pp-m0: /tmp/vagrant-puppet/modules-0
[default] Running provisioner: Vagrant::Provisioners::Puppet...
[default] Running Puppet with /tmp/vagrant-puppet/manifests/quantal64.pp...
stdin: is not a tty
notice: /Stage[main]//Node[default]/Exec[apt_update]/returns: executed successfully
notice: /Stage[main]/Docker/Exec[fetch-docker]/returns: executed successfully
notice: /Stage[main]/Docker/Package[lxc]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Docker/Exec[fetch-go]/returns: executed successfully
notice: /Stage[main]/Docker/Exec[copy-docker-bin]/returns: executed successfully
notice: /Stage[main]/Docker/Exec[debootstrap]/returns: executed successfully
notice: /Stage[main]/Docker/File[/etc/init/dockerd.conf]/ensure: defined content as '{md5}78a593d38dd9919af14d8f0545ac95e9'
notice: /Stage[main]/Docker/Service[dockerd]/ensure: ensure changed 'stopped' to 'running'
notice: Finished catalog run in 329.74 seconds
```
When this has successfully completed, you should be able to get into your new system with `vagrant ssh` and use `docker`:
```bash
$ vagrant ssh
Welcome to Ubuntu 12.10 (GNU/Linux 3.5.0-17-generic x86_64)
* Documentation: https://help.ubuntu.com/
Last login: Sun Feb 3 19:37:37 2013
vagrant@vagrant-ubuntu-12:~$ DOCKER=localhost:4242 docker help
Usage: docker COMMAND [arg...]
A self-sufficient runtime for linux containers.
Commands:
run Run a command in a container
ps Display a list of containers
pull Download a tarball and create a container from it
put Upload a tarball and create a container from it
rm Remove containers
wait Wait for the state of a container to change
stop Stop a running container
logs Fetch the logs of a container
diff Inspect changes on a container's filesystem
commit Save the state of a container
attach Attach to the standard inputs and outputs of a running container
info Display system-wide information
tar Stream the contents of a container as a tar archive
web Generate a web UI
attach Attach to a running container
```

4
client/client.go
View file

@ -1,8 +1,8 @@
package client package client
import ( import (
"github.com/dotcloud/docker/rcli"
"github.com/dotcloud/docker/future" "github.com/dotcloud/docker/future"
"github.com/dotcloud/docker/rcli"
"io" "io"
"io/ioutil" "io/ioutil"
"log" "log"
@ -112,7 +112,7 @@ func InteractiveMode(scripts ...string) error {
return err return err
} }
io.WriteString(rcfile, "enable -n help\n") io.WriteString(rcfile, "enable -n help\n")
os.Setenv("PATH", tmp + ":" + os.Getenv("PATH")) os.Setenv("PATH", tmp+":"+os.Getenv("PATH"))
os.Setenv("PS1", "\\h docker> ") os.Setenv("PS1", "\\h docker> ")
shell := exec.Command("/bin/bash", append([]string{"--rcfile", rcfile.Name()}, scripts...)...) shell := exec.Command("/bin/bash", append([]string{"--rcfile", rcfile.Name()}, scripts...)...)
shell.Stdin = os.Stdin shell.Stdin = os.Stdin

172
client/term.go
View file

@ -15,7 +15,6 @@ type Termios struct {
Ospeed uintptr Ospeed uintptr
} }
const ( const (
// Input flags // Input flags
inpck = 0x010 inpck = 0x010
@ -35,113 +34,110 @@ const (
) )
const ( const (
HUPCL = 0x4000 HUPCL = 0x4000
ICANON = 0x100 ICANON = 0x100
ICRNL = 0x100 ICRNL = 0x100
IEXTEN = 0x400 IEXTEN = 0x400
BRKINT = 0x2 BRKINT = 0x2
CFLUSH = 0xf CFLUSH = 0xf
CLOCAL = 0x8000 CLOCAL = 0x8000
CREAD = 0x800 CREAD = 0x800
CS5 = 0x0 CS5 = 0x0
CS6 = 0x100 CS6 = 0x100
CS7 = 0x200 CS7 = 0x200
CS8 = 0x300 CS8 = 0x300
CSIZE = 0x300 CSIZE = 0x300
CSTART = 0x11 CSTART = 0x11
CSTATUS = 0x14 CSTATUS = 0x14
CSTOP = 0x13 CSTOP = 0x13
CSTOPB = 0x400 CSTOPB = 0x400
CSUSP = 0x1a CSUSP = 0x1a
IGNBRK = 0x1 IGNBRK = 0x1
IGNCR = 0x80 IGNCR = 0x80
IGNPAR = 0x4 IGNPAR = 0x4
IMAXBEL = 0x2000 IMAXBEL = 0x2000
INLCR = 0x40 INLCR = 0x40
INPCK = 0x10 INPCK = 0x10
ISIG = 0x80 ISIG = 0x80
ISTRIP = 0x20 ISTRIP = 0x20
IUTF8 = 0x4000 IUTF8 = 0x4000
IXANY = 0x800 IXANY = 0x800
IXOFF = 0x400 IXOFF = 0x400
IXON = 0x200 IXON = 0x200
NOFLSH = 0x80000000 NOFLSH = 0x80000000
OCRNL = 0x10 OCRNL = 0x10
OFDEL = 0x20000 OFDEL = 0x20000
OFILL = 0x80 OFILL = 0x80
ONLCR = 0x2 ONLCR = 0x2
ONLRET = 0x40 ONLRET = 0x40
ONOCR = 0x20 ONOCR = 0x20
ONOEOT = 0x8 ONOEOT = 0x8
OPOST = 0x1 OPOST = 0x1
RENB = 0x1000 RENB = 0x1000
PARMRK = 0x8 PARMRK = 0x8
PARODD = 0x2000 PARODD = 0x2000
TOSTOP = 0x400000 TOSTOP = 0x400000
VDISCARD = 0xf VDISCARD = 0xf
VDSUSP = 0xb VDSUSP = 0xb
VEOF = 0x0 VEOF = 0x0
VEOL = 0x1 VEOL = 0x1
VEOL2 = 0x2 VEOL2 = 0x2
VERASE = 0x3 VERASE = 0x3
VINTR = 0x8 VINTR = 0x8
VKILL = 0x5 VKILL = 0x5
VLNEXT = 0xe VLNEXT = 0xe
VMIN = 0x10 VMIN = 0x10
VQUIT = 0x9 VQUIT = 0x9
VREPRINT = 0x6 VREPRINT = 0x6
VSTART = 0xc VSTART = 0xc
VSTATUS = 0x12 VSTATUS = 0x12
VSTOP = 0xd VSTOP = 0xd
VSUSP = 0xa VSUSP = 0xa
VT0 = 0x0 VT0 = 0x0
VT1 = 0x10000 VT1 = 0x10000
VTDLY = 0x10000 VTDLY = 0x10000
VTIME = 0x11 VTIME = 0x11
ECHO = 0x00000008 ECHO = 0x00000008
PENDIN = 0x20000000 PENDIN = 0x20000000
) )
type State struct { type State struct {
termios Termios termios Termios
} }
// IsTerminal returns true if the given file descriptor is a terminal. // IsTerminal returns true if the given file descriptor is a terminal.
func IsTerminal(fd int) bool { func IsTerminal(fd int) bool {
var termios Termios var termios Termios
_, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(fd), uintptr(getTermios), uintptr(unsafe.Pointer(&termios)), 0, 0, 0) _, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(fd), uintptr(getTermios), uintptr(unsafe.Pointer(&termios)), 0, 0, 0)
return err == 0 return err == 0
} }
// MakeRaw put the terminal connected to the given file descriptor into raw // MakeRaw put the terminal connected to the given file descriptor into raw
// mode and returns the previous state of the terminal so that it can be // mode and returns the previous state of the terminal so that it can be
// restored. // restored.
func MakeRaw(fd int) (*State, error) { func MakeRaw(fd int) (*State, error) {
var oldState State var oldState State
if _, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(fd), uintptr(getTermios), uintptr(unsafe.Pointer(&oldState.termios)), 0, 0, 0); err != 0 { if _, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(fd), uintptr(getTermios), uintptr(unsafe.Pointer(&oldState.termios)), 0, 0, 0); err != 0 {
return nil, err return nil, err
} }
newState := oldState.termios newState := oldState.termios
newState.Iflag &^= ISTRIP | INLCR | IGNCR | IXON | IXOFF newState.Iflag &^= ISTRIP | INLCR | IGNCR | IXON | IXOFF
newState.Iflag |= ICRNL newState.Iflag |= ICRNL
newState.Oflag |= ONLCR newState.Oflag |= ONLCR
newState.Lflag &^= ECHO | ICANON | ISIG newState.Lflag &^= ECHO | ICANON | ISIG
if _, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(fd), uintptr(setTermios), uintptr(unsafe.Pointer(&newState)), 0, 0, 0); err != 0 { if _, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(fd), uintptr(setTermios), uintptr(unsafe.Pointer(&newState)), 0, 0, 0); err != 0 {
return nil, err return nil, err
} }
return &oldState, nil return &oldState, nil
} }
// Restore restores the terminal connected to the given file descriptor to a // Restore restores the terminal connected to the given file descriptor to a
// previous state. // previous state.
func Restore(fd int, state *State) error { func Restore(fd int, state *State) error {
_, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(fd), uintptr(setTermios), uintptr(unsafe.Pointer(&state.termios)), 0, 0, 0) _, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(fd), uintptr(setTermios), uintptr(unsafe.Pointer(&state.termios)), 0, 0, 0)
return err return err
} }

165
container.go
View file

@ -1,7 +1,7 @@
package docker package docker
import ( import (
"bytes" "./fs"
"encoding/json" "encoding/json"
"errors" "errors"
"github.com/kr/pty" "github.com/kr/pty"
@ -11,10 +11,9 @@ import (
"os" "os"
"os/exec" "os/exec"
"path" "path"
"strings" "strconv"
"syscall" "syscall"
"time" "time"
"./fs"
) )
var sysInitPath string var sysInitPath string
@ -35,7 +34,11 @@ type Container struct {
Config *Config Config *Config
Mountpoint *fs.Mountpoint Mountpoint *fs.Mountpoint
State *State State *State
Image string Image string
network *NetworkInterface
networkManager *NetworkManager
NetworkSettings *NetworkSettings
SysInitPath string SysInitPath string
lxcConfigPath string lxcConfigPath string
@ -45,40 +48,61 @@ type Container struct {
stdin io.ReadCloser stdin io.ReadCloser
stdinPipe io.WriteCloser stdinPipe io.WriteCloser
stdoutLog *bytes.Buffer stdoutLog *os.File
stderrLog *bytes.Buffer stderrLog *os.File
} }
type Config struct { type Config struct {
Hostname string Hostname string
User string User string
Ram int64 Ram int64
Ports []int
Tty bool // Attach standard streams to a tty, including stdin if it is not closed. Tty bool // Attach standard streams to a tty, including stdin if it is not closed.
OpenStdin bool // Open stdin OpenStdin bool // Open stdin
} }
func createContainer(id string, root string, command string, args []string, image *fs.Image, config *Config) (*Container, error) { type NetworkSettings struct {
IpAddress string
IpPrefixLen int
Gateway string
PortMapping map[string]string
}
func createContainer(id string, root string, command string, args []string, image *fs.Image, config *Config, netManager *NetworkManager) (*Container, error) {
mountpoint, err := image.Mountpoint(path.Join(root, "rootfs"), path.Join(root, "rw")) mountpoint, err := image.Mountpoint(path.Join(root, "rootfs"), path.Join(root, "rw"))
if err != nil { if err != nil {
return nil, err return nil, err
} }
container := &Container{ container := &Container{
Id: id, Id: id,
Root: root, Root: root,
Created: time.Now(), Created: time.Now(),
Path: command, Path: command,
Args: args, Args: args,
Config: config, Config: config,
Image: image.Id, Image: image.Id,
Mountpoint: mountpoint, Mountpoint: mountpoint,
State: newState(), State: newState(),
networkManager: netManager,
SysInitPath: sysInitPath, NetworkSettings: &NetworkSettings{},
lxcConfigPath: path.Join(root, "config.lxc"), SysInitPath: sysInitPath,
stdout: newWriteBroadcaster(), lxcConfigPath: path.Join(root, "config.lxc"),
stderr: newWriteBroadcaster(), stdout: newWriteBroadcaster(),
stdoutLog: new(bytes.Buffer), stderr: newWriteBroadcaster(),
stderrLog: new(bytes.Buffer), }
if err := os.Mkdir(root, 0700); err != nil {
return nil, err
}
// Setup logging of stdout and stderr to disk
if stdoutLog, err := os.OpenFile(path.Join(container.Root, id+"-stdout.log"), os.O_RDWR|os.O_APPEND|os.O_CREATE, 0600); err != nil {
return nil, err
} else {
container.stdoutLog = stdoutLog
}
if stderrLog, err := os.OpenFile(path.Join(container.Root, id+"-stderr.log"), os.O_RDWR|os.O_APPEND|os.O_CREATE, 0600); err != nil {
return nil, err
} else {
container.stderrLog = stderrLog
} }
if container.Config.OpenStdin { if container.Config.OpenStdin {
container.stdin, container.stdinPipe = io.Pipe() container.stdin, container.stdinPipe = io.Pipe()
@ -88,36 +112,43 @@ func createContainer(id string, root string, command string, args []string, imag
container.stdout.AddWriter(NopWriteCloser(container.stdoutLog)) container.stdout.AddWriter(NopWriteCloser(container.stdoutLog))
container.stderr.AddWriter(NopWriteCloser(container.stderrLog)) container.stderr.AddWriter(NopWriteCloser(container.stderrLog))
if err := os.Mkdir(root, 0700); err != nil {
return nil, err
}
/*if err := container.Filesystem.createMountPoints(); err != nil {
return nil, err
}*/
if err := container.save(); err != nil { if err := container.save(); err != nil {
return nil, err return nil, err
} }
return container, nil return container, nil
} }
func loadContainer(containerPath string) (*Container, error) { func loadContainer(containerPath string, netManager *NetworkManager) (*Container, error) {
data, err := ioutil.ReadFile(path.Join(containerPath, "config.json")) data, err := ioutil.ReadFile(path.Join(containerPath, "config.json"))
if err != nil { if err != nil {
return nil, err return nil, err
} }
container := &Container{ container := &Container{
stdout: newWriteBroadcaster(), stdout: newWriteBroadcaster(),
stderr: newWriteBroadcaster(), stderr: newWriteBroadcaster(),
stdoutLog: new(bytes.Buffer), lxcConfigPath: path.Join(containerPath, "config.lxc"),
stderrLog: new(bytes.Buffer), networkManager: netManager,
lxcConfigPath: path.Join(containerPath, "config.lxc"), NetworkSettings: &NetworkSettings{},
} }
// Load container settings
if err := json.Unmarshal(data, container); err != nil { if err := json.Unmarshal(data, container); err != nil {
return nil, err return nil, err
} }
// if err := container.Filesystem.createMountPoints(); err != nil {
// return nil, err // Setup logging of stdout and stderr to disk
// } if stdoutLog, err := os.OpenFile(path.Join(container.Root, container.Id+"-stdout.log"), os.O_RDWR|os.O_APPEND|os.O_CREATE, 0600); err != nil {
return nil, err
} else {
container.stdoutLog = stdoutLog
}
if stderrLog, err := os.OpenFile(path.Join(container.Root, container.Id+"-stderr.log"), os.O_RDWR|os.O_APPEND|os.O_CREATE, 0600); err != nil {
return nil, err
} else {
container.stderrLog = stderrLog
}
container.stdout.AddWriter(NopWriteCloser(container.stdoutLog))
container.stderr.AddWriter(NopWriteCloser(container.stderrLog))
if container.Config.OpenStdin { if container.Config.OpenStdin {
container.stdin, container.stdinPipe = io.Pipe() container.stdin, container.stdinPipe = io.Pipe()
} else { } else {
@ -270,6 +301,9 @@ func (container *Container) Start() error {
if err := container.Mountpoint.EnsureMounted(); err != nil { if err := container.Mountpoint.EnsureMounted(); err != nil {
return err return err
} }
if err := container.allocateNetwork(); err != nil {
return err
}
if err := container.generateLXCConfig(); err != nil { if err := container.generateLXCConfig(); err != nil {
return err return err
} }
@ -279,11 +313,19 @@ func (container *Container) Start() error {
"--", "--",
"/sbin/init", "/sbin/init",
} }
// Networking
params = append(params, "-g", container.network.Gateway.String())
// User
if container.Config.User != "" { if container.Config.User != "" {
params = append(params, "-u", container.Config.User) params = append(params, "-u", container.Config.User)
} }
// Program
params = append(params, "--", container.Path) params = append(params, "--", container.Path)
params = append(params, container.Args...) params = append(params, container.Args...)
container.cmd = exec.Command("/usr/bin/lxc-start", params...) container.cmd = exec.Command("/usr/bin/lxc-start", params...)
var err error var err error
@ -337,7 +379,11 @@ func (container *Container) StdoutPipe() (io.ReadCloser, error) {
} }
func (container *Container) StdoutLog() io.Reader { func (container *Container) StdoutLog() io.Reader {
return strings.NewReader(container.stdoutLog.String()) r, err := os.Open(container.stdoutLog.Name())
if err != nil {
return nil
}
return r
} }
func (container *Container) StderrPipe() (io.ReadCloser, error) { func (container *Container) StderrPipe() (io.ReadCloser, error) {
@ -347,7 +393,39 @@ func (container *Container) StderrPipe() (io.ReadCloser, error) {
} }
func (container *Container) StderrLog() io.Reader { func (container *Container) StderrLog() io.Reader {
return strings.NewReader(container.stderrLog.String()) r, err := os.Open(container.stderrLog.Name())
if err != nil {
return nil
}
return r
}
func (container *Container) allocateNetwork() error {
iface, err := container.networkManager.Allocate()
if err != nil {
return err
}
container.NetworkSettings.PortMapping = make(map[string]string)
for _, port := range container.Config.Ports {
if extPort, err := iface.AllocatePort(port); err != nil {
iface.Release()
return err
} else {
container.NetworkSettings.PortMapping[strconv.Itoa(port)] = strconv.Itoa(extPort)
}
}
container.network = iface
container.NetworkSettings.IpAddress = iface.IPNet.IP.String()
container.NetworkSettings.IpPrefixLen, _ = iface.IPNet.Mask.Size()
container.NetworkSettings.Gateway = iface.Gateway.String()
return nil
}
func (container *Container) releaseNetwork() error {
err := container.network.Release()
container.network = nil
container.NetworkSettings = &NetworkSettings{}
return err
} }
func (container *Container) monitor() { func (container *Container) monitor() {
@ -356,6 +434,9 @@ func (container *Container) monitor() {
exitCode := container.cmd.ProcessState.Sys().(syscall.WaitStatus).ExitStatus() exitCode := container.cmd.ProcessState.Sys().(syscall.WaitStatus).ExitStatus()
// Cleanup // Cleanup
if err := container.releaseNetwork(); err != nil {
log.Printf("%v: Failed to release network: %v", container.Id, err)
}
container.stdout.Close() container.stdout.Close()
container.stderr.Close() container.stderr.Close()
if err := container.Mountpoint.Umount(); err != nil { if err := container.Mountpoint.Umount(); err != nil {
@ -422,11 +503,13 @@ func (container *Container) Restart() error {
return nil return nil
} }
func (container *Container) Wait() { // Wait blocks until the container stops running, then returns its exit code.
func (container *Container) Wait() int {
for container.State.Running { for container.State.Running {
container.State.wait() container.State.wait()
} }
return container.State.ExitCode
} }
func (container *Container) WaitTimeout(timeout time.Duration) error { func (container *Container) WaitTimeout(timeout time.Duration) error {

50
container_test.go
View file

@ -4,6 +4,7 @@ import (
"fmt" "fmt"
"io" "io"
"io/ioutil" "io/ioutil"
"sort"
"strings" "strings"
"testing" "testing"
"time" "time"
@ -513,6 +514,55 @@ func TestTty(t *testing.T) {
} }
} }
func TestEnv(t *testing.T) {
docker, err := newTestDocker()
if err != nil {
t.Fatal(err)
}
container, err := docker.Create(
"env_test",
"/usr/bin/env",
[]string{},
GetTestImage(docker),
&Config{},
)
if err != nil {
t.Fatal(err)
}
defer docker.Destroy(container)
stdout, err := container.StdoutPipe()
if err != nil {
t.Fatal(err)
}
defer stdout.Close()
if err := container.Start(); err != nil {
t.Fatal(err)
}
container.Wait()
output, err := ioutil.ReadAll(stdout)
if err != nil {
t.Fatal(err)
}
actualEnv := strings.Split(string(output), "\n")
if actualEnv[len(actualEnv)-1] == "" {
actualEnv = actualEnv[:len(actualEnv)-1]
}
sort.Strings(actualEnv)
goodEnv := []string{
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"HOME=/",
}
sort.Strings(goodEnv)
if len(goodEnv) != len(actualEnv) {
t.Fatalf("Wrong environment: should be %d variables, not: '%s'\n", len(goodEnv), strings.Join(actualEnv, ", "))
}
for i := range goodEnv {
if actualEnv[i] != goodEnv[i] {
t.Fatalf("Wrong environment variable: should be %s, not %s", goodEnv[i], actualEnv[i])
}
}
}
func BenchmarkRunSequencial(b *testing.B) { func BenchmarkRunSequencial(b *testing.B) {
docker, err := newTestDocker() docker, err := newTestDocker()
if err != nil { if err != nil {

29
docker.go
View file

@ -1,6 +1,7 @@
package docker package docker
import ( import (
"./fs"
"container/list" "container/list"
"fmt" "fmt"
"io/ioutil" "io/ioutil"
@ -8,14 +9,14 @@ import (
"os" "os"
"path" "path"
"sort" "sort"
"./fs"
) )
type Docker struct { type Docker struct {
root string root string
repository string repository string
containers *list.List containers *list.List
Store *fs.Store networkManager *NetworkManager
Store *fs.Store
} }
func (docker *Docker) List() []*Container { func (docker *Docker) List() []*Container {
@ -53,7 +54,8 @@ func (docker *Docker) Create(id string, command string, args []string, image *fs
return nil, fmt.Errorf("Container %v already exists", id) return nil, fmt.Errorf("Container %v already exists", id)
} }
root := path.Join(docker.repository, id) root := path.Join(docker.repository, id)
container, err := createContainer(id, root, command, args, image, config)
container, err := createContainer(id, root, command, args, image, config, docker.networkManager)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@ -92,7 +94,7 @@ func (docker *Docker) restore() error {
return err return err
} }
for _, v := range dir { for _, v := range dir {
container, err := loadContainer(path.Join(docker.repository, v.Name())) container, err := loadContainer(path.Join(docker.repository, v.Name()), docker.networkManager)
if err != nil { if err != nil {
log.Printf("Failed to load container %v: %v", v.Name(), err) log.Printf("Failed to load container %v: %v", v.Name(), err)
continue continue
@ -112,12 +114,17 @@ func NewFromDirectory(root string) (*Docker, error) {
if err != nil { if err != nil {
return nil, err return nil, err
} }
netManager, err := newNetworkManager(networkBridgeIface)
if err != nil {
return nil, err
}
docker := &Docker{ docker := &Docker{
root: root, root: root,
repository: path.Join(root, "containers"), repository: path.Join(root, "containers"),
containers: list.New(), containers: list.New(),
Store: store, Store: store,
networkManager: netManager,
} }
if err := os.MkdirAll(docker.repository, 0700); err != nil && !os.IsExist(err) { if err := os.MkdirAll(docker.repository, 0700); err != nil && !os.IsExist(err) {

3
docker/docker.go
View file

@ -2,10 +2,10 @@ package main
import ( import (
"flag" "flag"
"github.com/dotcloud/docker/client"
"log" "log"
"os" "os"
"path" "path"
"github.com/dotcloud/docker/client"
) )
func main() { func main() {
@ -27,4 +27,3 @@ func main() {
} }
} }
} }

8
docker_test.go
View file

@ -1,17 +1,17 @@
package docker package docker
import ( import (
"./fs"
"io"
"io/ioutil" "io/ioutil"
"log" "log"
"os" "os"
"testing" "testing"
"io"
"./fs"
) )
const testLayerPath string = "/var/lib/docker/docker-ut.tar" const testLayerPath string = "/var/lib/docker/docker-ut.tar"
func layerArchive(tarfile string) (io.Reader, error) { func layerArchive(tarfile string) (io.Reader, error) {
// FIXME: need to close f somewhere // FIXME: need to close f somewhere
f, err := os.Open(tarfile) f, err := os.Open(tarfile)
if err != nil { if err != nil {
@ -57,7 +57,7 @@ func newTestDocker() (*Docker, error) {
return docker, nil return docker, nil
} }
func GetTestImage(docker *Docker) (*fs.Image) { func GetTestImage(docker *Docker) *fs.Image {
imgs, err := docker.Store.Images() imgs, err := docker.Store.Images()
if err != nil { if err != nil {
panic(err) panic(err)

73
examples/pybuilder Executable file
View file

@ -0,0 +1,73 @@
#!/usr/bin/env docker -i
# Uncomment to debug:
#set -x
export NORAW=1
IMG=shykes/pybuilder:11d4f58638a72935
if [ $# -lt 3 ]; then
echo "Usage: $0 build|run USER/REPO REV"
echo "Example usage:"
echo ""
echo " REV=7d5f035432fe1453eea389b0f1b02a2a93c8009e"
echo " $0 build shykes/helloflask \$REV"
echo " $0 run shykes/helloflask \$REV"
echo ""
exit 1
fi
CMD=$1
FORCE=0
if [ "$2" = "-f" ]; then
FORCE=1
shift
fi
REPO=$2
REV=$3
BUILD_IMAGE=builds/github.com/$REPO/$REV
if [ "$CMD" = "build" ]; then
if [ ! -z "`images -q $BUILD_IMAGE`" ]; then
if [ "$FORCE" -ne 1 ]; then
echo "$BUILD_IMAGE already exists"
exit
fi
fi
# Allocate a TTY to work around python's aggressive buffering of stdout
BUILD_JOB=`run -t $IMG /usr/local/bin/buildapp http://github.com/$REPO/archive/$REV.tar.gz`
if [ -z "$BUILD_JOB" ]; then
echo "Build failed"
exit 1
fi
if attach $BUILD_JOB ; then
BUILD_STATUS=`docker wait $BUILD_JOB`
if [ -z "$BUILD_STATUS" -o "$BUILD_STATUS" != 0 ]; then
echo "Build failed"
exit 1
fi
else
echo "Build failed"
exit 1
fi
commit $BUILD_JOB $BUILD_IMAGE
echo "Build saved at $BUILD_IMAGE"
elif [ "$CMD" = "run" ]; then
RUN_JOB=`run $BUILD_IMAGE /usr/local/bin/runapp`
if [ -z "$RUN_JOB" ]; then
echo "Run failed"
exit 1
fi
attach $RUN_JOB
fi

16
fake/fake.go
View file

@ -1,20 +1,19 @@
package fake package fake
import ( import (
"bytes"
"math/rand"
"io"
"archive/tar" "archive/tar"
"os/exec" "bytes"
"github.com/kr/pty" "github.com/kr/pty"
"io"
"math/rand"
"os/exec"
) )
func FakeTar() (io.Reader, error) { func FakeTar() (io.Reader, error) {
content := []byte("Hello world!\n") content := []byte("Hello world!\n")
buf := new(bytes.Buffer) buf := new(bytes.Buffer)
tw := tar.NewWriter(buf) tw := tar.NewWriter(buf)
for _, name := range []string {"/etc/postgres/postgres.conf", "/etc/passwd", "/var/log/postgres/postgres.conf"} { for _, name := range []string{"hello", "etc/postgres/postgres.conf", "etc/passwd", "var/log/postgres/postgres.conf"} {
hdr := new(tar.Header) hdr := new(tar.Header)
hdr.Size = int64(len(content)) hdr.Size = int64(len(content))
hdr.Name = name hdr.Name = name
@ -27,7 +26,6 @@ func FakeTar() (io.Reader, error) {
return buf, nil return buf, nil
} }
func WriteFakeTar(dst io.Writer) error { func WriteFakeTar(dst io.Writer) error {
if data, err := FakeTar(); err != nil { if data, err := FakeTar(); err != nil {
return err return err
@ -37,7 +35,6 @@ func WriteFakeTar(dst io.Writer) error {
return nil return nil
} }
func RandomBytesChanged() uint { func RandomBytesChanged() uint {
return uint(rand.Int31n(24 * 1024 * 1024)) return uint(rand.Int31n(24 * 1024 * 1024))
} }
@ -54,7 +51,6 @@ func ContainerRunning() bool {
return false return false
} }
func StartCommand(cmd *exec.Cmd, interactive bool) (io.WriteCloser, io.ReadCloser, error) { func StartCommand(cmd *exec.Cmd, interactive bool) (io.WriteCloser, io.ReadCloser, error) {
if interactive { if interactive {
term, err := pty.Start(cmd) term, err := pty.Start(cmd)
@ -76,5 +72,3 @@ func StartCommand(cmd *exec.Cmd, interactive bool) (io.WriteCloser, io.ReadClose
} }
return stdin, stdout, nil return stdin, stdout, nil
} }

73
fs/archive.go Normal file
View file

@ -0,0 +1,73 @@
package fs
import (
"errors"
"io"
"io/ioutil"
"os/exec"
)
type Compression uint32
const (
Uncompressed Compression = iota
Bzip2
Gzip
)
func (compression *Compression) Flag() string {
switch *compression {
case Bzip2:
return "j"
case Gzip:
return "z"
}
return ""
}
func Tar(path string, compression Compression) (io.Reader, error) {
cmd := exec.Command("bsdtar", "-f", "-", "-C", path, "-c"+compression.Flag(), ".")
return CmdStream(cmd)
}
func Untar(archive io.Reader, path string) error {
cmd := exec.Command("bsdtar", "-f", "-", "-C", path, "-x")
cmd.Stdin = archive
output, err := cmd.CombinedOutput()
if err != nil {
return errors.New(err.Error() + ": " + string(output))
}
return nil
}
func CmdStream(cmd *exec.Cmd) (io.Reader, error) {
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, err
}
stderr, err := cmd.StderrPipe()
if err != nil {
return nil, err
}
pipeR, pipeW := io.Pipe()
go func() {
_, err := io.Copy(pipeW, stdout)
if err != nil {
pipeW.CloseWithError(err)
}
errText, e := ioutil.ReadAll(stderr)
if e != nil {
errText = []byte("(...couldn't fetch stderr: " + e.Error() + ")")
}
if err := cmd.Wait(); err != nil {
// FIXME: can this block if stderr outputs more than the size of StderrPipe()'s buffer?
pipeW.CloseWithError(errors.New(err.Error() + ": " + string(errText)))
} else {
pipeW.Close()
}
}()
if err := cmd.Start(); err != nil {
return nil, err
}
return pipeR, nil
}

54
fs/archive_test.go Normal file
View file

@ -0,0 +1,54 @@
package fs
import (
"io/ioutil"
"os"
"os/exec"
"testing"
)
func TestCmdStreamBad(t *testing.T) {
badCmd := exec.Command("/bin/sh", "-c", "echo hello; echo >&2 error couldn\\'t reverse the phase pulser; exit 1")
out, err := CmdStream(badCmd)
if err != nil {
t.Fatalf("Failed to start command: " + err.Error())
}
if output, err := ioutil.ReadAll(out); err == nil {
t.Fatalf("Command should have failed")
} else if err.Error() != "exit status 1: error couldn't reverse the phase pulser\n" {
t.Fatalf("Wrong error value (%s)", err.Error())
} else if s := string(output); s != "hello\n" {
t.Fatalf("Command output should be '%s', not '%s'", "hello\\n", output)
}
}
func TestCmdStreamGood(t *testing.T) {
cmd := exec.Command("/bin/sh", "-c", "echo hello; exit 0")
out, err := CmdStream(cmd)
if err != nil {
t.Fatal(err)
}
if output, err := ioutil.ReadAll(out); err != nil {
t.Fatalf("Command should not have failed (err=%s)", err)
} else if s := string(output); s != "hello\n" {
t.Fatalf("Command output should be '%s', not '%s'", "hello\\n", output)
}
}
func TestTarUntar(t *testing.T) {
archive, err := Tar(".", Uncompressed)
if err != nil {
t.Fatal(err)
}
tmp, err := ioutil.TempDir("", "docker-test-untar")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmp)
if err := Untar(archive, tmp); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(tmp); err != nil {
t.Fatalf("Error stating %s: %s", tmp, err.Error())
}
}

82
fs/layers.go
View file

@ -1,29 +1,20 @@
package fs package fs
import ( import (
"../future"
"errors" "errors"
"path" "fmt"
"path/filepath"
"io" "io"
"io/ioutil" "io/ioutil"
"os" "os"
"os/exec" "path"
"fmt" "path/filepath"
"../future"
) )
type LayerStore struct { type LayerStore struct {
Root string Root string
} }
type Compression uint32
const (
Uncompressed Compression = iota
Bzip2
Gzip
)
func NewLayerStore(root string) (*LayerStore, error) { func NewLayerStore(root string) (*LayerStore, error) {
abspath, err := filepath.Abs(root) abspath, err := filepath.Abs(root)
if err != nil { if err != nil {
@ -80,10 +71,9 @@ func (store *LayerStore) Init() error {
return os.Mkdir(store.Root, 0700) return os.Mkdir(store.Root, 0700)
} }
func (store *LayerStore) Mktemp() (string, error) { func (store *LayerStore) Mktemp() (string, error) {
tmpName := future.RandomId() tmpName := future.RandomId()
tmpPath := path.Join(store.Root, "tmp-" + tmpName) tmpPath := path.Join(store.Root, "tmp-"+tmpName)
if err := os.Mkdir(tmpPath, 0700); err != nil { if err := os.Mkdir(tmpPath, 0700); err != nil {
return "", err return "", err
} }
@ -94,54 +84,42 @@ func (store *LayerStore) layerPath(id string) string {
return path.Join(store.Root, id) return path.Join(store.Root, id)
} }
func (store *LayerStore) AddLayer(id string, archive Archive) (string, error) {
func (store *LayerStore) AddLayer(id string, archive Archive, stderr io.Writer, compression Compression) (string, error) {
if _, err := os.Stat(store.layerPath(id)); err == nil { if _, err := os.Stat(store.layerPath(id)); err == nil {
return "", errors.New("Layer already exists: " + id) return "", fmt.Errorf("Layer already exists: %v", id)
} }
errors := make(chan error)
// Untar
tmp, err := store.Mktemp() tmp, err := store.Mktemp()
defer os.RemoveAll(tmp) defer os.RemoveAll(tmp)
if err != nil { if err != nil {
return "", errors.New(fmt.Sprintf("Mktemp failed: %s", err)) return "", fmt.Errorf("Mktemp failed: %s", err)
} }
extractFlags := "-x"
if compression == Bzip2 {
extractFlags += "j"
} else if compression == Gzip {
extractFlags += "z"
}
untarCmd := exec.Command("tar", "-C", tmp, extractFlags)
untarW, err := untarCmd.StdinPipe()
if err != nil {
return "", errors.New(fmt.Sprintf("Could not obtain stdin pipe: %s", err))
}
untarStderr, err := untarCmd.StderrPipe()
if err != nil {
return "", errors.New(fmt.Sprintf("Could not obtain stderr pipe: %s", err))
}
go io.Copy(stderr, untarStderr)
untarStdout, err := untarCmd.StdoutPipe()
if err != nil {
return "", errors.New(fmt.Sprintf("Could not obtain stdout pipe: %s", err))
}
go io.Copy(stderr, untarStdout)
untarCmd.Start()
job_copy := future.Go(func() error {
_, err := io.Copy(untarW, archive)
untarW.Close()
return err
})
if err := untarCmd.Wait(); err != nil { untarR, untarW := io.Pipe()
return "", errors.New(fmt.Sprintf("Error while waiting for untar command to complete: %s", err)) go func() {
errors <- Untar(untarR, tmp)
}()
_, err = io.Copy(untarW, archive)
untarW.Close()
if err != nil {
return "", err
} }
if err := <-job_copy; err != nil { // Wait for goroutines
return "", errors.New(fmt.Sprintf("Error while copying: %s", err)) for i := 0; i < 1; i += 1 {
select {
case err := <-errors:
{
if err != nil {
return "", err
}
}
}
} }
layer := store.layerPath(id) layer := store.layerPath(id)
if !store.Exists(id) { if !store.Exists(id) {
if err := os.Rename(tmp, layer); err != nil { if err := os.Rename(tmp, layer); err != nil {
return "", errors.New(fmt.Sprintf("Could not rename temp dir to layer %s: %s", layer, err)) return "", fmt.Errorf("Could not rename temp dir to layer %s: %s", layer, err)
} }
} }
return layer, nil return layer, nil

15
fs/layers_test.go
View file

@ -1,14 +1,12 @@
package fs package fs
import ( import (
"io/ioutil"
"testing"
"os"
"github.com/dotcloud/docker/fake" "github.com/dotcloud/docker/fake"
"io/ioutil"
"os"
"testing"
) )
func TestLayersInit(t *testing.T) { func TestLayersInit(t *testing.T) {
store := tempStore(t) store := tempStore(t)
defer os.RemoveAll(store.Root) defer os.RemoveAll(store.Root)
@ -25,7 +23,7 @@ func TestLayersInit(t *testing.T) {
func TestAddLayer(t *testing.T) { func TestAddLayer(t *testing.T) {
store := tempStore(t) store := tempStore(t)
defer os.RemoveAll(store.Root) defer os.RemoveAll(store.Root)
layer, err := store.AddLayer("foo", testArchive(t), os.Stderr, Uncompressed) layer, err := store.AddLayer("foo", testArchive(t))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@ -46,15 +44,14 @@ func TestAddLayer(t *testing.T) {
func TestAddLayerDuplicate(t *testing.T) { func TestAddLayerDuplicate(t *testing.T) {
store := tempStore(t) store := tempStore(t)
defer os.RemoveAll(store.Root) defer os.RemoveAll(store.Root)
if _, err := store.AddLayer("foobar123", testArchive(t), os.Stderr, Uncompressed); err != nil { if _, err := store.AddLayer("foobar123", testArchive(t)); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if _, err := store.AddLayer("foobar123", testArchive(t), os.Stderr, Uncompressed); err == nil { if _, err := store.AddLayer("foobar123", testArchive(t)); err == nil {
t.Fatalf("Creating duplicate layer should fail") t.Fatalf("Creating duplicate layer should fail")
} }
} }
/* /*
* HELPER FUNCTIONS * HELPER FUNCTIONS
*/ */

7
fs/store.go
View file

@ -10,9 +10,9 @@ import (
"io" "io"
"os" "os"
"path" "path"
"path/filepath"
"syscall" "syscall"
"time" "time"
"path/filepath"
) )
type Store struct { type Store struct {
@ -121,7 +121,7 @@ func (store *Store) Create(layerData Archive, parent *Image, pth, comment string
} }
// FIXME: we shouldn't have to pass os.Stderr to AddLayer()... // FIXME: we shouldn't have to pass os.Stderr to AddLayer()...
// FIXME: Archive should contain compression info. For now we only support uncompressed. // FIXME: Archive should contain compression info. For now we only support uncompressed.
_, err := store.layers.AddLayer(img.Id, layerData, os.Stderr, Uncompressed) _, err := store.layers.AddLayer(img.Id, layerData)
if err != nil { if err != nil {
return nil, errors.New(fmt.Sprintf("Could not add layer: %s", err)) return nil, errors.New(fmt.Sprintf("Could not add layer: %s", err))
} }
@ -168,7 +168,6 @@ type Image struct {
store *Store `db:"-"` store *Store `db:"-"`
} }
func (image *Image) Copy(pth string) (*Image, error) { func (image *Image) Copy(pth string) (*Image, error) {
if err := image.store.orm.Insert(&Path{Path: pth, Image: image.Id}); err != nil { if err := image.store.orm.Insert(&Path{Path: pth, Image: image.Id}); err != nil {
return nil, err return nil, err
@ -198,7 +197,7 @@ func (image *Image) Mountpoint(root, rw string) (*Mountpoint, error) {
func (image *Image) layers() ([]string, error) { func (image *Image) layers() ([]string, error) {
var list []string var list []string
var err error var err error
currentImg := image currentImg := image
for currentImg != nil { for currentImg != nil {
if layer := image.store.layers.Get(image.Id); layer != "" { if layer := image.store.layers.Get(image.Id); layer != "" {

63
future/future.go
View file

@ -1,12 +1,13 @@
package future package future
import ( import (
"crypto/sha256"
"io"
"fmt"
"time"
"bytes" "bytes"
"crypto/sha256"
"fmt"
"io"
"math/rand" "math/rand"
"os/exec"
"time"
) )
func Seed() { func Seed() {
@ -30,18 +31,18 @@ func HumanDuration(d time.Duration) string {
return "About a minute" return "About a minute"
} else if minutes < 60 { } else if minutes < 60 {
return fmt.Sprintf("%d minutes", minutes) return fmt.Sprintf("%d minutes", minutes)
} else if hours := int(d.Hours()); hours == 1{ } else if hours := int(d.Hours()); hours == 1 {
return "About an hour" return "About an hour"
} else if hours < 48 { } else if hours < 48 {
return fmt.Sprintf("%d hours", hours) return fmt.Sprintf("%d hours", hours)
} else if hours < 24 * 7 * 2 { } else if hours < 24*7*2 {
return fmt.Sprintf("%d days", hours / 24) return fmt.Sprintf("%d days", hours/24)
} else if hours < 24 * 30 * 3 { } else if hours < 24*30*3 {
return fmt.Sprintf("%d weeks", hours / 24 / 7) return fmt.Sprintf("%d weeks", hours/24/7)
} else if hours < 24 * 365 * 2 { } else if hours < 24*365*2 {
return fmt.Sprintf("%d months", hours / 24 / 30) return fmt.Sprintf("%d months", hours/24/30)
} }
return fmt.Sprintf("%d years", d.Hours() / 24 / 365) return fmt.Sprintf("%d years", d.Hours()/24/365)
} }
func randomBytes() io.Reader { func randomBytes() io.Reader {
@ -61,3 +62,41 @@ func Go(f func() error) chan error {
return ch return ch
} }
// Pv wraps an io.Reader such that it is passed through unchanged,
// but logs the number of bytes copied (comparable to the unix command pv)
func Pv(src io.Reader, info io.Writer) io.Reader {
var totalBytes int
data := make([]byte, 2048)
r, w := io.Pipe()
go func() {
for {
if n, err := src.Read(data); err != nil {
w.CloseWithError(err)
return
} else {
totalBytes += n
fmt.Fprintf(info, "--> %d bytes\n", totalBytes)
if _, err = w.Write(data[:n]); err != nil {
return
}
}
}
}()
return r
}
// Curl makes an http request by executing the unix command 'curl', and returns
// the body of the response. If `stderr` is not nil, a progress bar will be
// written to it.
func Curl(url string, stderr io.Writer) (io.Reader, error) {
curl := exec.Command("curl", "-#", "-L", url)
output, err := curl.StdoutPipe()
if err != nil {
return nil, err
}
curl.Stderr = stderr
if err := curl.Start(); err != nil {
return nil, err
}
return output, nil
}

73
image/archive.go Normal file
View file

@ -0,0 +1,73 @@
package image
import (
"errors"
"io"
"io/ioutil"
"os/exec"
)
type Compression uint32
const (
Uncompressed Compression = iota
Bzip2
Gzip
)
func (compression *Compression) Flag() string {
switch *compression {
case Bzip2:
return "j"
case Gzip:
return "z"
}
return ""
}
func Tar(path string, compression Compression) (io.Reader, error) {
cmd := exec.Command("bsdtar", "-f", "-", "-C", path, "-c"+compression.Flag(), ".")
return CmdStream(cmd)
}
func Untar(archive io.Reader, path string) error {
cmd := exec.Command("bsdtar", "-f", "-", "-C", path, "-x")
cmd.Stdin = archive
output, err := cmd.CombinedOutput()
if err != nil {
return errors.New(err.Error() + ": " + string(output))
}
return nil
}
func CmdStream(cmd *exec.Cmd) (io.Reader, error) {
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, err
}
stderr, err := cmd.StderrPipe()
if err != nil {
return nil, err
}
pipeR, pipeW := io.Pipe()
go func() {
_, err := io.Copy(pipeW, stdout)
if err != nil {
pipeW.CloseWithError(err)
}
errText, e := ioutil.ReadAll(stderr)
if e != nil {
errText = []byte("(...couldn't fetch stderr: " + e.Error() + ")")
}
if err := cmd.Wait(); err != nil {
// FIXME: can this block if stderr outputs more than the size of StderrPipe()'s buffer?
pipeW.CloseWithError(errors.New(err.Error() + ": " + string(errText)))
} else {
pipeW.Close()
}
}()
if err := cmd.Start(); err != nil {
return nil, err
}
return pipeR, nil
}

54
image/archive_test.go Normal file
View file

@ -0,0 +1,54 @@
package image
import (
"io/ioutil"
"os"
"os/exec"
"testing"
)
func TestCmdStreamBad(t *testing.T) {
badCmd := exec.Command("/bin/sh", "-c", "echo hello; echo >&2 error couldn\\'t reverse the phase pulser; exit 1")
out, err := CmdStream(badCmd)
if err != nil {
t.Fatalf("Failed to start command: " + err.Error())
}
if output, err := ioutil.ReadAll(out); err == nil {
t.Fatalf("Command should have failed")
} else if err.Error() != "exit status 1: error couldn't reverse the phase pulser\n" {
t.Fatalf("Wrong error value (%s)", err.Error())
} else if s := string(output); s != "hello\n" {
t.Fatalf("Command output should be '%s', not '%s'", "hello\\n", output)
}
}
func TestCmdStreamGood(t *testing.T) {
cmd := exec.Command("/bin/sh", "-c", "echo hello; exit 0")
out, err := CmdStream(cmd)
if err != nil {
t.Fatal(err)
}
if output, err := ioutil.ReadAll(out); err != nil {
t.Fatalf("Command should not have failed (err=%s)", err)
} else if s := string(output); s != "hello\n" {
t.Fatalf("Command output should be '%s', not '%s'", "hello\\n", output)
}
}
func TestTarUntar(t *testing.T) {
archive, err := Tar(".", Uncompressed)
if err != nil {
t.Fatal(err)
}
tmp, err := ioutil.TempDir("", "docker-test-untar")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmp)
if err := Untar(archive, tmp); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(tmp); err != nil {
t.Fatalf("Error stating %s: %s", tmp, err.Error())
}
}

95
image/image.go
View file

@ -1,27 +1,26 @@
package image package image
import ( import (
"encoding/json"
"errors"
"github.com/dotcloud/docker/future"
"io" "io"
"io/ioutil" "io/ioutil"
"encoding/json" "os"
"time"
"path" "path"
"path/filepath" "path/filepath"
"errors" "regexp"
"sort" "sort"
"os"
"github.com/dotcloud/docker/future"
"strings" "strings"
"time"
) )
type Store struct { type Store struct {
*Index *Index
Root string Root string
Layers *LayerStore Layers *LayerStore
} }
func New(root string) (*Store, error) { func New(root string) (*Store, error) {
abspath, err := filepath.Abs(root) abspath, err := filepath.Abs(root)
if err != nil { if err != nil {
@ -38,22 +37,16 @@ func New(root string) (*Store, error) {
return nil, err return nil, err
} }
return &Store{ return &Store{
Root: abspath, Root: abspath,
Index: NewIndex(path.Join(root, "index.json")), Index: NewIndex(path.Join(root, "index.json")),
Layers: layers, Layers: layers,
}, nil }, nil
} }
type Compression uint32 // Import creates a new image from the contents of `archive` and registers it in the store as `name`.
// If `parent` is not nil, it will registered as the parent of the new image.
const ( func (store *Store) Import(name string, archive io.Reader, parent *Image) (*Image, error) {
Uncompressed Compression = iota layer, err := store.Layers.AddLayer(archive)
Bzip2
Gzip
)
func (store *Store) Import(name string, archive io.Reader, stderr io.Writer, parent *Image, compression Compression) (*Image, error) {
layer, err := store.Layers.AddLayer(archive, stderr, compression)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@ -79,20 +72,19 @@ func (store *Store) Create(name string, source string, layers ...string) (*Image
return image, nil return image, nil
} }
// Index // Index
type Index struct { type Index struct {
Path string Path string
ByName map[string]*History ByName map[string]*History
ById map[string]*Image ById map[string]*Image
} }
func NewIndex(path string) *Index { func NewIndex(path string) *Index {
return &Index{ return &Index{
Path: path, Path: path,
ByName: make(map[string]*History), ByName: make(map[string]*History),
ById: make(map[string]*Image), ById: make(map[string]*Image),
} }
} }
@ -218,11 +210,36 @@ func (index *Index) Delete(name string) error {
return nil return nil
} }
// DeleteMatch deletes all images whose name matches `pattern`
func (index *Index) DeleteMatch(pattern string) error {
// Load
if err := index.load(); err != nil {
return err
}
for name, history := range index.ByName {
if match, err := regexp.MatchString(pattern, name); err != nil {
return err
} else if match {
// Remove from index lookup
for _, image := range *history {
delete(index.ById, image.Id)
}
// Remove from name lookup
delete(index.ByName, name)
}
}
// Save
if err := index.save(); err != nil {
return err
}
return nil
}
func (index *Index) Names() []string { func (index *Index) Names() []string {
if err := index.load(); err != nil { if err := index.load(); err != nil {
return []string{} return []string{}
} }
var names[]string var names []string
for name := range index.ByName { for name := range index.ByName {
names = append(names, name) names = append(names, name)
} }
@ -285,23 +302,23 @@ func (history *History) Add(image *Image) {
func (history *History) Del(id string) { func (history *History) Del(id string) {
for idx, image := range *history { for idx, image := range *history {
if image.Id == id { if image.Id == id {
*history = append((*history)[:idx], (*history)[idx + 1:]...) *history = append((*history)[:idx], (*history)[idx+1:]...)
} }
} }
} }
type Image struct { type Image struct {
Id string // Globally unique identifier Id string // Globally unique identifier
Layers []string // Absolute paths Layers []string // Absolute paths
Created time.Time Created time.Time
Parent string Parent string
} }
func (image *Image) IdParts() (string, string) { func (image *Image) IdParts() (string, string) {
if len(image.Id) < 8 { if len(image.Id) < 8 {
return "", image.Id return "", image.Id
} }
hash := image.Id[len(image.Id)-8:len(image.Id)] hash := image.Id[len(image.Id)-8 : len(image.Id)]
name := image.Id[:len(image.Id)-9] name := image.Id[:len(image.Id)-9]
return name, hash return name, hash
} }
@ -322,7 +339,7 @@ func generateImageId(name string, layers []string) (string, error) {
for _, layer := range layers { for _, layer := range layers {
ids += path.Base(layer) ids += path.Base(layer)
} }
if h, err := future.ComputeId(strings.NewReader(ids)); err != nil { if h, err := future.ComputeId(strings.NewReader(ids)); err != nil {
return "", err return "", err
} else { } else {
hash = h hash = h
@ -337,9 +354,9 @@ func NewImage(name string, layers []string, parent string) (*Image, error) {
return nil, err return nil, err
} }
return &Image{ return &Image{
Id: id, Id: id,
Layers: layers, Layers: layers,
Created: time.Now(), Created: time.Now(),
Parent: parent, Parent: parent,
}, nil }, nil
} }

47
image/layers_test.go Normal file
View file

@ -0,0 +1,47 @@
package image
import (
"bytes"
"github.com/dotcloud/docker/fake"
"github.com/dotcloud/docker/future"
"io/ioutil"
"os"
"testing"
)
func TestAddLayer(t *testing.T) {
tmp, err := ioutil.TempDir("", "docker-test-image")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmp)
store, err := NewLayerStore(tmp)
if err != nil {
t.Fatal(err)
}
archive, err := fake.FakeTar()
if err != nil {
t.Fatal(err)
}
layer, err := store.AddLayer(archive)
if err != nil {
t.Fatal(err)
}
if _, err := os.Stat(layer); err != nil {
t.Fatalf("Error testing for existence of layer: %s\n", err.Error())
}
}
func TestComputeId(t *testing.T) {
id1, err := future.ComputeId(bytes.NewBufferString("hello world\n"))
if err != nil {
t.Fatal(err)
}
id2, err := future.ComputeId(bytes.NewBufferString("foo bar\n"))
if err != nil {
t.Fatal(err)
}
if id1 == id2 {
t.Fatalf("Identical checksums for difference content (%s == %s)", id1, id2)
}
}

34
install.sh Normal file
View file

@ -0,0 +1,34 @@
# This script is meant for quick & easy install via 'curl URL-OF-SCRIPPT | bash'
# Courtesy of Jeff Lindsay <progrium@gmail.com>
cd /tmp
echo "Ensuring dependencies are installed..."
apt-get --yes install lxc wget bsdtar 2>&1 > /dev/null
echo "Downloading docker binary..."
wget -q https://dl.dropbox.com/u/20637798/docker.tar.gz 2>&1 > /dev/null
tar -xf docker.tar.gz 2>&1 > /dev/null
echo "Installing into /usr/local/bin..."
mv docker/docker /usr/local/bin
mv dockerd/dockerd /usr/local/bin
if [[ -f /etc/init/dockerd.conf ]]
then
echo "Upstart script already exists."
else
echo "Creating /etc/init/dockerd.conf..."
echo "exec /usr/local/bin/dockerd" > /etc/init/dockerd.conf
fi
echo "Restarting dockerd..."
restart dockerd > /dev/null
echo "Cleaning up..."
rmdir docker
rmdir dockerd
rm docker.tar.gz
echo "Finished!"
echo

12
lxc_template.go
View file

@ -14,12 +14,12 @@ lxc.utsname = {{.Id}}
#lxc.aa_profile = unconfined #lxc.aa_profile = unconfined
# network configuration # network configuration
#lxc.network.type = veth lxc.network.type = veth
#lxc.network.flags = up lxc.network.flags = up
#lxc.network.link = br0 lxc.network.link = lxcbr0
#lxc.network.name = eth0 # Internal container network interface name lxc.network.name = eth0
#lxc.network.mtu = 1500 lxc.network.mtu = 1500
#lxc.network.ipv4 = {ip_address}/{ip_prefix_len} lxc.network.ipv4 = {{.NetworkSettings.IpAddress}}/{{.NetworkSettings.IpPrefixLen}}
# root filesystem # root filesystem
{{$ROOTFS := .Mountpoint.Root}} {{$ROOTFS := .Mountpoint.Root}}

1
mount_linux.go
View file

@ -2,7 +2,6 @@ package docker
import "syscall" import "syscall"
func mount(source string, target string, fstype string, flags uintptr, data string) (err error) { func mount(source string, target string, fstype string, flags uintptr, data string) (err error) {
return syscall.Mount(source, target, fstype, flags, data) return syscall.Mount(source, target, fstype, flags, data)
} }

356
network.go Normal file
View file

@ -0,0 +1,356 @@
package docker
import (
"bytes"
"encoding/binary"
"errors"
"fmt"
"log"
"net"
"os/exec"
"strconv"
"strings"
)
const (
networkBridgeIface = "lxcbr0"
portRangeStart = 49153
portRangeEnd = 65535
)
// Calculates the first and last IP addresses in an IPNet
func networkRange(network *net.IPNet) (net.IP, net.IP) {
netIP := network.IP.To4()
firstIP := netIP.Mask(network.Mask)
lastIP := net.IPv4(0, 0, 0, 0).To4()
for i := 0; i < len(lastIP); i++ {
lastIP[i] = netIP[i] | ^network.Mask[i]
}
return firstIP, lastIP
}
// Converts a 4 bytes IP into a 32 bit integer
func ipToInt(ip net.IP) (int32, error) {
buf := bytes.NewBuffer(ip.To4())
var n int32
if err := binary.Read(buf, binary.BigEndian, &n); err != nil {
return 0, err
}
return n, nil
}
// Converts 32 bit integer into a 4 bytes IP address
func intToIp(n int32) (net.IP, error) {
var buf bytes.Buffer
if err := binary.Write(&buf, binary.BigEndian, &n); err != nil {
return net.IP{}, err
}
ip := net.IPv4(0, 0, 0, 0).To4()
for i := 0; i < net.IPv4len; i++ {
ip[i] = buf.Bytes()[i]
}
return ip, nil
}
// Given a netmask, calculates the number of available hosts
func networkSize(mask net.IPMask) (int32, error) {
m := net.IPv4Mask(0, 0, 0, 0)
for i := 0; i < net.IPv4len; i++ {
m[i] = ^mask[i]
}
buf := bytes.NewBuffer(m)
var n int32
if err := binary.Read(buf, binary.BigEndian, &n); err != nil {
return 0, err
}
return n + 1, nil
}
// Wrapper around the iptables command
func iptables(args ...string) error {
if err := exec.Command("/sbin/iptables", args...).Run(); err != nil {
return fmt.Errorf("iptables failed: iptables %v", strings.Join(args, " "))
}
return nil
}
// Return the IPv4 address of a network interface
func getIfaceAddr(name string) (net.Addr, error) {
iface, err := net.InterfaceByName(name)
if err != nil {
return nil, err
}
addrs, err := iface.Addrs()
if err != nil {
return nil, err
}
var addrs4 []net.Addr
for _, addr := range addrs {
ip := (addr.(*net.IPNet)).IP
if ip4 := ip.To4(); len(ip4) == net.IPv4len {
addrs4 = append(addrs4, addr)
}
}
switch {
case len(addrs4) == 0:
return nil, fmt.Errorf("Interface %v has no IP addresses", name)
case len(addrs4) > 1:
return nil, fmt.Errorf("Interface %v has more than 1 IPv4 address", name)
}
return addrs4[0], nil
}
// Port mapper takes care of mapping external ports to containers by setting
// up iptables rules.
// It keeps track of all mappings and is able to unmap at will
type PortMapper struct {
mapping map[int]net.TCPAddr
}
func (mapper *PortMapper) cleanup() error {
// Ignore errors - This could mean the chains were never set up
iptables("-t", "nat", "-D", "PREROUTING", "-j", "DOCKER")
iptables("-t", "nat", "-F", "DOCKER")
iptables("-t", "nat", "-X", "DOCKER")
mapper.mapping = make(map[int]net.TCPAddr)
return nil
}
func (mapper *PortMapper) setup() error {
if err := iptables("-t", "nat", "-N", "DOCKER"); err != nil {
return errors.New("Unable to setup port networking: Failed to create DOCKER chain")
}
if err := iptables("-t", "nat", "-A", "PREROUTING", "-j", "DOCKER"); err != nil {
return errors.New("Unable to setup port networking: Failed to inject docker in PREROUTING chain")
}
return nil
}
func (mapper *PortMapper) iptablesForward(rule string, port int, dest net.TCPAddr) error {
return iptables("-t", "nat", rule, "DOCKER", "-p", "tcp", "--dport", strconv.Itoa(port),
"-j", "DNAT", "--to-destination", net.JoinHostPort(dest.IP.String(), strconv.Itoa(dest.Port)))
}
func (mapper *PortMapper) Map(port int, dest net.TCPAddr) error {
if err := mapper.iptablesForward("-A", port, dest); err != nil {
return err
}
mapper.mapping[port] = dest
return nil
}
func (mapper *PortMapper) Unmap(port int) error {
dest, ok := mapper.mapping[port]
if !ok {
return errors.New("Port is not mapped")
}
if err := mapper.iptablesForward("-D", port, dest); err != nil {
return err
}
delete(mapper.mapping, port)
return nil
}
func newPortMapper() (*PortMapper, error) {
mapper := &PortMapper{}
if err := mapper.cleanup(); err != nil {
return nil, err
}
if err := mapper.setup(); err != nil {
return nil, err
}
return mapper, nil
}
// Port allocator: Atomatically allocate and release networking ports
type PortAllocator struct {
ports chan (int)
}
func (alloc *PortAllocator) populate(start, end int) {
alloc.ports = make(chan int, end-start)
for port := start; port < end; port++ {
alloc.ports <- port
}
}
func (alloc *PortAllocator) Acquire() (int, error) {
select {
case port := <-alloc.ports:
return port, nil
default:
return -1, errors.New("No more ports available")
}
return -1, nil
}
func (alloc *PortAllocator) Release(port int) error {
select {
case alloc.ports <- port:
return nil
default:
return errors.New("Too many ports have been released")
}
return nil
}
func newPortAllocator(start, end int) (*PortAllocator, error) {
allocator := &PortAllocator{}
allocator.populate(start, end)
return allocator, nil
}
// IP allocator: Atomatically allocate and release networking ports
type IPAllocator struct {
network *net.IPNet
queue chan (net.IP)
}
func (alloc *IPAllocator) populate() error {
firstIP, _ := networkRange(alloc.network)
size, err := networkSize(alloc.network.Mask)
if err != nil {
return err
}
// The queue size should be the network size - 3
// -1 for the network address, -1 for the broadcast address and
// -1 for the gateway address
alloc.queue = make(chan net.IP, size-3)
for i := int32(1); i < size-1; i++ {
ipNum, err := ipToInt(firstIP)
if err != nil {
return err
}
ip, err := intToIp(ipNum + int32(i))
if err != nil {
return err
}
// Discard the network IP (that's the host IP address)
if ip.Equal(alloc.network.IP) {
continue
}
alloc.queue <- ip
}
return nil
}
func (alloc *IPAllocator) Acquire() (net.IP, error) {
select {
case ip := <-alloc.queue:
return ip, nil
default:
return net.IP{}, errors.New("No more IP addresses available")
}
return net.IP{}, nil
}
func (alloc *IPAllocator) Release(ip net.IP) error {
select {
case alloc.queue <- ip:
return nil
default:
return errors.New("Too many IP addresses have been released")
}
return nil
}
func newIPAllocator(network *net.IPNet) (*IPAllocator, error) {
alloc := &IPAllocator{
network: network,
}
if err := alloc.populate(); err != nil {
return nil, err
}
return alloc, nil
}
// Network interface represents the networking stack of a container
type NetworkInterface struct {
IPNet net.IPNet
Gateway net.IP
manager *NetworkManager
extPorts []int
}
// Allocate an external TCP port and map it to the interface
func (iface *NetworkInterface) AllocatePort(port int) (int, error) {
extPort, err := iface.manager.portAllocator.Acquire()
if err != nil {
return -1, err
}
if err := iface.manager.portMapper.Map(extPort, net.TCPAddr{iface.IPNet.IP, port}); err != nil {
iface.manager.portAllocator.Release(extPort)
return -1, err
}
iface.extPorts = append(iface.extPorts, extPort)
return extPort, nil
}
// Release: Network cleanup - release all resources
func (iface *NetworkInterface) Release() error {
for _, port := range iface.extPorts {
if err := iface.manager.portMapper.Unmap(port); err != nil {
log.Printf("Unable to unmap port %v: %v", port, err)
}
if err := iface.manager.portAllocator.Release(port); err != nil {
log.Printf("Unable to release port %v: %v", port, err)
}
}
return iface.manager.ipAllocator.Release(iface.IPNet.IP)
}
// Network Manager manages a set of network interfaces
// Only *one* manager per host machine should be used
type NetworkManager struct {
bridgeIface string
bridgeNetwork *net.IPNet
ipAllocator *IPAllocator
portAllocator *PortAllocator
portMapper *PortMapper
}
// Allocate a network interface
func (manager *NetworkManager) Allocate() (*NetworkInterface, error) {
ip, err := manager.ipAllocator.Acquire()
if err != nil {
return nil, err
}
iface := &NetworkInterface{
IPNet: net.IPNet{ip, manager.bridgeNetwork.Mask},
Gateway: manager.bridgeNetwork.IP,
manager: manager,
}
return iface, nil
}
func newNetworkManager(bridgeIface string) (*NetworkManager, error) {
addr, err := getIfaceAddr(bridgeIface)
if err != nil {
return nil, err
}
network := addr.(*net.IPNet)
ipAllocator, err := newIPAllocator(network)
if err != nil {
return nil, err
}
portAllocator, err := newPortAllocator(portRangeStart, portRangeEnd)
if err != nil {
return nil, err
}
portMapper, err := newPortMapper()
manager := &NetworkManager{
bridgeIface: bridgeIface,
bridgeNetwork: network,
ipAllocator: ipAllocator,
portAllocator: portAllocator,
portMapper: portMapper,
}
return manager, nil
}

130
network_test.go Normal file
View file

@ -0,0 +1,130 @@
package docker
import (
"net"
"testing"
)
func TestNetworkRange(t *testing.T) {
// Simple class C test
_, network, _ := net.ParseCIDR("192.168.0.1/24")
first, last := networkRange(network)
if !first.Equal(net.ParseIP("192.168.0.0")) {
t.Error(first.String())
}
if !last.Equal(net.ParseIP("192.168.0.255")) {
t.Error(last.String())
}
if size, err := networkSize(network.Mask); err != nil || size != 256 {
t.Error(size, err)
}
// Class A test
_, network, _ = net.ParseCIDR("10.0.0.1/8")
first, last = networkRange(network)
if !first.Equal(net.ParseIP("10.0.0.0")) {
t.Error(first.String())
}
if !last.Equal(net.ParseIP("10.255.255.255")) {
t.Error(last.String())
}
if size, err := networkSize(network.Mask); err != nil || size != 16777216 {
t.Error(size, err)
}
// Class A, random IP address
_, network, _ = net.ParseCIDR("10.1.2.3/8")
first, last = networkRange(network)
if !first.Equal(net.ParseIP("10.0.0.0")) {
t.Error(first.String())
}
if !last.Equal(net.ParseIP("10.255.255.255")) {
t.Error(last.String())
}
// 32bit mask
_, network, _ = net.ParseCIDR("10.1.2.3/32")
first, last = networkRange(network)
if !first.Equal(net.ParseIP("10.1.2.3")) {
t.Error(first.String())
}
if !last.Equal(net.ParseIP("10.1.2.3")) {
t.Error(last.String())
}
if size, err := networkSize(network.Mask); err != nil || size != 1 {
t.Error(size, err)
}
// 31bit mask
_, network, _ = net.ParseCIDR("10.1.2.3/31")
first, last = networkRange(network)
if !first.Equal(net.ParseIP("10.1.2.2")) {
t.Error(first.String())
}
if !last.Equal(net.ParseIP("10.1.2.3")) {
t.Error(last.String())
}
if size, err := networkSize(network.Mask); err != nil || size != 2 {
t.Error(size, err)
}
// 26bit mask
_, network, _ = net.ParseCIDR("10.1.2.3/26")
first, last = networkRange(network)
if !first.Equal(net.ParseIP("10.1.2.0")) {
t.Error(first.String())
}
if !last.Equal(net.ParseIP("10.1.2.63")) {
t.Error(last.String())
}
if size, err := networkSize(network.Mask); err != nil || size != 64 {
t.Error(size, err)
}
}
func TestConversion(t *testing.T) {
ip := net.ParseIP("127.0.0.1")
i, err := ipToInt(ip)
if err != nil {
t.Fatal(err)
}
if i == 0 {
t.Fatal("converted to zero")
}
conv, err := intToIp(i)
if err != nil {
t.Fatal(err)
}
if !ip.Equal(conv) {
t.Error(conv.String())
}
}
func TestIPAllocator(t *testing.T) {
gwIP, n, _ := net.ParseCIDR("127.0.0.1/29")
alloc, err := newIPAllocator(&net.IPNet{gwIP, n.Mask})
if err != nil {
t.Fatal(err)
}
var lastIP net.IP
for i := 0; i < 5; i++ {
ip, err := alloc.Acquire()
if err != nil {
t.Fatal(err)
}
lastIP = ip
}
ip, err := alloc.Acquire()
if err == nil {
t.Fatal("There shouldn't be any IP addresses at this point")
}
// Release 1 IP
alloc.Release(lastIP)
ip, err = alloc.Acquire()
if err != nil {
t.Fatal(err)
}
if !ip.Equal(lastIP) {
t.Fatal(ip.String())
}
}

2
puppet/modules/docker/manifests/init.pp
View file

@ -8,7 +8,7 @@ class docker {
Package { ensure => "installed" } Package { ensure => "installed" }
package { ["lxc", "debootstrap", "wget"]: } package { ["lxc", "debootstrap", "wget", "bsdtar"]: }
exec { "debootstrap" : exec { "debootstrap" :
require => Package["debootstrap"], require => Package["debootstrap"],

112
server/server.go
View file

@ -1,20 +1,21 @@
package server package server
import ( import (
".."
"../fs"
"../future"
"../rcli"
"bufio" "bufio"
"bytes" "bytes"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
".."
"../future"
"../fs"
"../rcli"
"io" "io"
"net/http" "net/http"
"net/url" "net/url"
"os" "os"
"path" "path"
"strconv"
"strings" "strings"
"sync" "sync"
"text/tabwriter" "text/tabwriter"
@ -43,6 +44,7 @@ func (srv *Server) Help() string {
{"ps", "Display a list of containers"}, {"ps", "Display a list of containers"},
{"pull", "Download a tarball and create a container from it"}, {"pull", "Download a tarball and create a container from it"},
{"put", "Upload a tarball and create a container from it"}, {"put", "Upload a tarball and create a container from it"},
{"port", "Lookup the public-facing port which is NAT-ed to PRIVATE_PORT"},
{"rm", "Remove containers"}, {"rm", "Remove containers"},
{"kill", "Kill a running container"}, {"kill", "Kill a running container"},
{"wait", "Wait for the state of a container to change"}, {"wait", "Wait for the state of a container to change"},
@ -53,6 +55,7 @@ func (srv *Server) Help() string {
{"diff", "Inspect changes on a container's filesystem"}, {"diff", "Inspect changes on a container's filesystem"},
{"commit", "Save the state of a container"}, {"commit", "Save the state of a container"},
{"attach", "Attach to the standard inputs and outputs of a running container"}, {"attach", "Attach to the standard inputs and outputs of a running container"},
{"wait", "Block until a container exits, then print its exit code"},
{"info", "Display system-wide information"}, {"info", "Display system-wide information"},
{"tar", "Stream the contents of a container as a tar archive"}, {"tar", "Stream the contents of a container as a tar archive"},
{"web", "Generate a web UI"}, {"web", "Generate a web UI"},
@ -63,6 +66,27 @@ func (srv *Server) Help() string {
return help return help
} }
// 'docker wait': block until a container stops
func (srv *Server) CmdWait(stdin io.ReadCloser, stdout io.Writer, args ...string) error {
cmd := rcli.Subcmd(stdout, "wait", "[OPTIONS] NAME", "Block until a container stops, then print its exit code.")
if err := cmd.Parse(args); err != nil {
cmd.Usage()
return nil
}
if cmd.NArg() < 1 {
cmd.Usage()
return nil
}
for _, name := range cmd.Args() {
if container := srv.containers.Get(name); container != nil {
fmt.Fprintln(stdout, container.Wait())
} else {
return errors.New("No such container: " + name)
}
}
return nil
}
// 'docker info': display system-wide information. // 'docker info': display system-wide information.
func (srv *Server) CmdInfo(stdin io.ReadCloser, stdout io.Writer, args ...string) error { func (srv *Server) CmdInfo(stdin io.ReadCloser, stdout io.Writer, args ...string) error {
fmt.Fprintf(stdout, "containers: %d\nversion: %s\nimages: %d\n", fmt.Fprintf(stdout, "containers: %d\nversion: %s\nimages: %d\n",
@ -269,8 +293,8 @@ func (srv *Server) CmdInspect(stdin io.ReadCloser, stdout io.Writer, args ...str
var obj interface{} var obj interface{}
if container := srv.containers.Get(name); container != nil { if container := srv.containers.Get(name); container != nil {
obj = container obj = container
//} else if image, err := srv.images.List(name); image != nil { //} else if image, err := srv.images.List(name); image != nil {
// obj = image // obj = image
} else { } else {
return errors.New("No such container or image: " + name) return errors.New("No such container or image: " + name)
} }
@ -288,9 +312,34 @@ func (srv *Server) CmdInspect(stdin io.ReadCloser, stdout io.Writer, args ...str
return nil return nil
} }
func (srv *Server) CmdPort(stdin io.ReadCloser, stdout io.Writer, args ...string) error {
cmd := rcli.Subcmd(stdout, "port", "[OPTIONS] CONTAINER PRIVATE_PORT", "Lookup the public-facing port which is NAT-ed to PRIVATE_PORT")
if err := cmd.Parse(args); err != nil {
cmd.Usage()
return nil
}
if cmd.NArg() != 2 {
cmd.Usage()
return nil
}
name := cmd.Arg(0)
privatePort := cmd.Arg(1)
if container := srv.containers.Get(name); container == nil {
return errors.New("No such container: " + name)
} else {
if frontend, exists := container.NetworkSettings.PortMapping[privatePort]; !exists {
return fmt.Errorf("No private port '%s' allocated on %s", privatePort, name)
} else {
fmt.Fprintln(stdout, frontend)
}
}
return nil
}
// 'docker rmi NAME' removes all images with the name NAME // 'docker rmi NAME' removes all images with the name NAME
// func (srv *Server) CmdRmi(stdin io.ReadCloser, stdout io.Writer, args ...string) error { // func (srv *Server) CmdRmi(stdin io.ReadCloser, stdout io.Writer, args ...string) error {
// cmd := rcli.Subcmd(stdout, "rmimage", "[OPTIONS] IMAGE", "Remove an image") // cmd := rcli.Subcmd(stdout, "rmimage", "[OPTIONS] IMAGE", "Remove an image")
// fl_regexp := cmd.Bool("r", false, "Use IMAGE as a regular expression instead of an exact name")
// if err := cmd.Parse(args); err != nil { // if err := cmd.Parse(args); err != nil {
// cmd.Usage() // cmd.Usage()
// return nil // return nil
@ -300,11 +349,17 @@ func (srv *Server) CmdInspect(stdin io.ReadCloser, stdout io.Writer, args ...str
// return nil // return nil
// } // }
// for _, name := range cmd.Args() { // for _, name := range cmd.Args() {
// image := srv.images.Find(name) // var err error
// if image == nil { // if *fl_regexp {
// return errors.New("No such image: " + name) // err = srv.images.DeleteMatch(name)
// } else {
// image := srv.images.Find(name)
// if image == nil {
// return errors.New("No such image: " + name)
// }
// err = srv.images.Delete(name)
// } // }
// if err := srv.images.Delete(name); err != nil { // if err != nil {
// return err // return err
// } // }
// } // }
@ -367,11 +422,18 @@ func (srv *Server) CmdPull(stdin io.ReadCloser, stdout io.Writer, args ...string
u.Host = "s3.amazonaws.com" u.Host = "s3.amazonaws.com"
u.Path = path.Join("/docker.io/images", u.Path) u.Path = path.Join("/docker.io/images", u.Path)
} }
fmt.Fprintf(stdout, "Downloading %s from %s...\n", name, u.String()) fmt.Fprintf(stdout, "Downloading from %s\n", u.String())
resp, err := http.Get(u.String()) // Download with curl (pretty progress bar)
// If curl is not available, fallback to http.Get()
archive, err := future.Curl(u.String(), stdout)
if err != nil { if err != nil {
return err if resp, err := http.Get(u.String()); err != nil {
return err
} else {
archive = resp.Body
}
} }
fmt.Fprintf(stdout, "Unpacking to %s\n", name)
img, err := srv.images.Create(resp.Body, nil, name, "") img, err := srv.images.Create(resp.Body, nil, name, "")
if err != nil { if err != nil {
return err return err
@ -668,10 +730,10 @@ func (srv *Server) CmdLogs(stdin io.ReadCloser, stdout io.Writer, args ...string
return errors.New("No such container: " + cmd.Arg(0)) return errors.New("No such container: " + cmd.Arg(0))
} }
func (srv *Server) CreateContainer(img *fs.Image, user string, tty bool, openStdin bool, comment string, cmd string, args ...string) (*docker.Container, error) { func (srv *Server) CreateContainer(img *fs.Image, ports []int, user string, tty bool, openStdin bool, comment string, cmd string, args ...string) (*docker.Container, error) {
id := future.RandomId()[:8] id := future.RandomId()[:8]
container, err := srv.containers.Create(id, cmd, args, img, container, err := srv.containers.Create(id, cmd, args, img,
&docker.Config{Hostname: id, User: user, Tty: tty, OpenStdin: openStdin}) &docker.Config{Hostname: id, Ports: ports, User: user, Tty: tty, OpenStdin: openStdin})
if err != nil { if err != nil {
return nil, err return nil, err
} }
@ -732,6 +794,22 @@ func (srv *Server) CmdAttach(stdin io.ReadCloser, stdout io.Writer, args ...stri
return nil return nil
} }
// Ports type - Used to parse multiple -p flags
type ports []int
func (p *ports) String() string {
return fmt.Sprint(*p)
}
func (p *ports) Set(value string) error {
port, err := strconv.Atoi(value)
if err != nil {
return fmt.Errorf("Invalid port: %v", value)
}
*p = append(*p, port)
return nil
}
func (srv *Server) CmdRun(stdin io.ReadCloser, stdout io.Writer, args ...string) error { func (srv *Server) CmdRun(stdin io.ReadCloser, stdout io.Writer, args ...string) error {
cmd := rcli.Subcmd(stdout, "run", "[OPTIONS] IMAGE COMMAND [ARG...]", "Run a command in a new container") cmd := rcli.Subcmd(stdout, "run", "[OPTIONS] IMAGE COMMAND [ARG...]", "Run a command in a new container")
fl_user := cmd.String("u", "", "Username or UID") fl_user := cmd.String("u", "", "Username or UID")
@ -739,6 +817,8 @@ func (srv *Server) CmdRun(stdin io.ReadCloser, stdout io.Writer, args ...string)
fl_stdin := cmd.Bool("i", false, "Keep stdin open even if not attached") fl_stdin := cmd.Bool("i", false, "Keep stdin open even if not attached")
fl_tty := cmd.Bool("t", false, "Allocate a pseudo-tty") fl_tty := cmd.Bool("t", false, "Allocate a pseudo-tty")
fl_comment := cmd.String("c", "", "Comment") fl_comment := cmd.String("c", "", "Comment")
var fl_ports ports
cmd.Var(&fl_ports, "p", "Map a network port to the container")
if err := cmd.Parse(args); err != nil { if err := cmd.Parse(args); err != nil {
return nil return nil
} }
@ -766,7 +846,7 @@ func (srv *Server) CmdRun(stdin io.ReadCloser, stdout io.Writer, args ...string)
return errors.New("No such image: " + name) return errors.New("No such image: " + name)
} }
// Create new container // Create new container
container, err := srv.CreateContainer(img, *fl_user, *fl_tty, *fl_stdin, *fl_comment, cmdline[0], cmdline[1:]...) container, err := srv.CreateContainer(img, fl_ports, *fl_user, *fl_tty, *fl_stdin, *fl_comment, cmdline[0], cmdline[1:]...)
if err != nil { if err != nil {
return errors.New("Error creating container: " + err.Error()) return errors.New("Error creating container: " + err.Error())
} }

22
sysinit.go
View file

@ -11,6 +11,17 @@ import (
"syscall" "syscall"
) )
// Setup networking
func setupNetworking(gw string) {
if gw == "" {
return
}
cmd := exec.Command("/sbin/route", "add", "default", "gw", gw)
if err := cmd.Run(); err != nil {
log.Fatalf("Unable to set up networking: %v", err)
}
}
// Takes care of dropping privileges to the desired user // Takes care of dropping privileges to the desired user
func changeUser(u string) { func changeUser(u string) {
if u == "" { if u == "" {
@ -41,6 +52,13 @@ func changeUser(u string) {
} }
} }
// Set the environment to a known, repeatable state
func setupEnv() {
os.Clearenv()
os.Setenv("HOME", "/")
os.Setenv("PATH", "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin")
}
func executeProgram(name string, args []string) { func executeProgram(name string, args []string) {
path, err := exec.LookPath(name) path, err := exec.LookPath(name)
if err != nil { if err != nil {
@ -62,8 +80,12 @@ func SysInit() {
os.Exit(1) os.Exit(1)
} }
var u = flag.String("u", "", "username or uid") var u = flag.String("u", "", "username or uid")
var gw = flag.String("g", "", "gateway address")
flag.Parse() flag.Parse()
setupNetworking(*gw)
changeUser(*u) changeUser(*u)
setupEnv()
executeProgram(flag.Arg(0), flag.Args()) executeProgram(flag.Arg(0), flag.Args())
} }

19
utils.go
View file

@ -17,25 +17,6 @@ func Trunc(s string, maxlen int) string {
return s[:maxlen] return s[:maxlen]
} }
// Tar generates a tar archive from a filesystem path, and returns it as a stream.
// Path must point to a directory.
func Tar(path string) (io.Reader, error) {
cmd := exec.Command("tar", "-C", path, "-c", ".")
output, err := cmd.StdoutPipe()
if err != nil {
return nil, err
}
if err := cmd.Start(); err != nil {
return nil, err
}
// FIXME: errors will not be passed because we don't wait for the command.
// Instead, consumers will hit EOF right away.
// This can be fixed by waiting for the process to exit, or for the first write
// on stdout, whichever comes first.
return output, nil
}
// Figure out the absolute path of our own binary // Figure out the absolute path of our own binary
func SelfPath() string { func SelfPath() string {
path, err := exec.LookPath(os.Args[0]) path, err := exec.LookPath(os.Args[0])