diff --git a/daemon/graphdriver/devmapper/README.md b/daemon/graphdriver/devmapper/README.md index e8e14094b0..4d2cad7b91 100644 --- a/daemon/graphdriver/devmapper/README.md +++ b/daemon/graphdriver/devmapper/README.md @@ -71,204 +71,14 @@ status information about the driver. * `Metadata loop file` file attached to `Metadata file`, if loopback device is used * `Library Version` from the libdevmapper used -### options +### About the devicemapper options The devicemapper backend supports some options that you can specify when starting the docker daemon using the `--storage-opt` flags. This uses the `dm` prefix and would be used something like `docker -d --storage-opt dm.foo=bar`. -Here is the list of supported options: - - * `dm.basesize` - - Specifies the size to use when creating the base device, which - limits the size of images and containers. The default value is - 10G. Note, thin devices are inherently "sparse", so a 10G device - which is mostly empty doesn't use 10 GB of space on the - pool. However, the filesystem will use more space for the empty - case the larger the device is. **Warning**: This value affects the - system-wide "base" empty filesystem that may already be - initialized and inherited by pulled images. Typically, a change - to this value will require additional steps to take effect: 1) - stop `docker -d`, 2) `rm -rf /var/lib/docker`, 3) start `docker -d`. - - Example use: - - ``docker -d --storage-opt dm.basesize=20G`` - - * `dm.loopdatasize` - - Specifies the size to use when creating the loopback file for the - "data" device which is used for the thin pool. The default size is - 100G. Note that the file is sparse, so it will not initially take - up this much space. - - Example use: - - ``docker -d --storage-opt dm.loopdatasize=200G`` - - * `dm.loopmetadatasize` - - Specifies the size to use when creating the loopback file for the - "metadadata" device which is used for the thin pool. The default size is - 2G. Note that the file is sparse, so it will not initially take - up this much space. - - Example use: - - ``docker -d --storage-opt dm.loopmetadatasize=4G`` - - * `dm.fs` - - Specifies the filesystem type to use for the base device. The supported - options are "ext4" and "xfs". The default is "ext4" - - Example use: - - ``docker -d --storage-opt dm.fs=xfs`` - - * `dm.mkfsarg` - - Specifies extra mkfs arguments to be used when creating the base device. - - Example use: - - ``docker -d --storage-opt "dm.mkfsarg=-O ^has_journal"`` - - * `dm.mountopt` - - Specifies extra mount options used when mounting the thin devices. - - Example use: - - ``docker -d --storage-opt dm.mountopt=nodiscard`` - - * `dm.thinpooldev` - - Specifies a custom blockdevice to use for the thin pool. - - If using a block device for device mapper storage, ideally lvm2 - would be used to create/manage the thin-pool volume that is then - handed to docker to exclusively create/manage the thin and thin - snapshot volumes needed for its containers. Managing the thin-pool - outside of docker makes for the most feature-rich method of having - docker utilize device mapper thin provisioning as the backing - storage for docker's containers. lvm2-based thin-pool management - feature highlights include: automatic or interactive thin-pool - resize support, dynamically change thin-pool features, automatic - thinp metadata checking when lvm2 activates the thin-pool, etc. - - Example use: - - ``docker -d --storage-opt dm.thinpooldev=/dev/mapper/thin-pool`` - - * `dm.datadev` - - Specifies a custom blockdevice to use for data for the thin pool. - - If using a block device for device mapper storage, ideally both - datadev and metadatadev should be specified to completely avoid - using the loopback device. - - Example use: - - ``docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1`` - - * `dm.metadatadev` - - Specifies a custom blockdevice to use for metadata for the thin - pool. - - For best performance the metadata should be on a different spindle - than the data, or even better on an SSD. - - If setting up a new metadata pool it is required to be valid. This - can be achieved by zeroing the first 4k to indicate empty - metadata, like this: - - ``dd if=/dev/zero of=$metadata_dev bs=4096 count=1`` - - Example use: - - ``docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1`` - - * `dm.blocksize` - - Specifies a custom blocksize to use for the thin pool. The default - blocksize is 64K. - - Example use: - - ``docker -d --storage-opt dm.blocksize=512K`` - - * `dm.blkdiscard` - - Enables or disables the use of blkdiscard when removing - devicemapper devices. This is enabled by default (only) if using - loopback devices and is required to resparsify the loopback file - on image/container removal. - - Disabling this on loopback can lead to *much* faster container - removal times, but will make the space used in /var/lib/docker - directory not be returned to the system for other use when - containers are removed. - - Example use: - - ``docker -d --storage-opt dm.blkdiscard=false`` - - * `dm.override_udev_sync_check` - - Overrides the `udev` synchronization checks between `devicemapper` and `udev`. - `udev` is the device manager for the Linux kernel. - - To view the `udev` sync support of a Docker daemon that is using the - `devicemapper` driver, run: - - $ docker info - [...] - Udev Sync Supported: true - [...] - - When `udev` sync support is `true`, then `devicemapper` and udev can - coordinate the activation and deactivation of devices for containers. - - When `udev` sync support is `false`, a race condition occurs between - the`devicemapper` and `udev` during create and cleanup. The race condition - results in errors and failures. (For information on these failures, see - [docker#4036](https://github.com/docker/docker/issues/4036)) - - To allow the `docker` daemon to start, regardless of `udev` sync not being - supported, set `dm.override_udev_sync_check` to true: - - $ docker -d --storage-opt dm.override_udev_sync_check=true - - When this value is `true`, the `devicemapper` continues and simply warns - you the errors are happening. - - > **Note**: The ideal is to pursue a `docker` daemon and environment that - > does support synchronizing with `udev`. For further discussion on this - > topic, see [docker#4036](https://github.com/docker/docker/issues/4036). - > Otherwise, set this flag for migrating existing Docker daemons to a - > daemon with a supported environment. - - * `dm.use_deferred_removal` - - Enables use of deferred device removal if libdm and kernel driver - support the mechanism. - - Deferred device removal means that if device is busy when devices is - being removed/deactivated, then a deferred removal is scheduled on - device. And devices automatically goes away when last user of device - exits. - - For example, when container exits, its associated thin device is - removed. If that devices has leaked into some other mount namespace - can can't be removed now, container exit will still be successful - and this option will just schedule device for deferred removal and - will not wait in a loop trying to remove a busy device. - - Example use: - - ``docker -d --storage-opt dm.use_deferred_removal=true`` - +These options are currently documented both in [the man +page](../../../man/docker.1.md) and in [the online +documentation](https://docs.docker.com/reference/commandline/daemon/#docker- +execdriver-option). If you add an options, update both the `man` page and the +documentation. diff --git a/docs/reference/commandline/daemon.md b/docs/reference/commandline/daemon.md index c66e9cd6c9..0cd9f734e0 100644 --- a/docs/reference/commandline/daemon.md +++ b/docs/reference/commandline/daemon.md @@ -178,7 +178,24 @@ Particular storage-driver can be configured with options specified with `--storage-opt` flags. Options for `devicemapper` are prefixed with `dm` and options for `zfs` start with `zfs`. -Currently supported options of `devicemapper`: +* `dm.thinpooldev` + + Specifies a custom block storage device to use for the thin pool. + + If using a block device for device mapper storage, it is best to use `lvm` + to create and manage the thin-pool volume. This volume is then handed to Docker + to exclusively create snapshot volumes needed for images and containers. + + Managing the thin-pool outside of Docker makes for the most feature-rich + method of having Docker utilize device mapper thin provisioning as the + backing storage for Docker's containers. The highlights of the lvm-based + thin-pool management feature include: automatic or interactive thin-pool + resize support, dynamically changing thin-pool features, automatic thinp + metadata checking when lvm activates the thin-pool, etc. + + Example use: + + docker -d --storage-opt dm.thinpooldev=/dev/mapper/thin-pool * `dm.basesize` @@ -188,9 +205,9 @@ Currently supported options of `devicemapper`: 10 GB of space on the pool. However, the filesystem will use more space for the empty case the larger the device is. - **Warning:** This value affects the system-wide "base" empty filesystem - that may already be initialized and inherited by pulled images. Typically, - a change to this value will require additional steps to take effect: + This value affects the system-wide "base" empty filesystem + that may already be initialized and inherited by pulled images. Typically, + a change to this value requires additional steps to take effect: $ sudo service docker stop $ sudo rm -rf /var/lib/docker @@ -202,9 +219,12 @@ Currently supported options of `devicemapper`: * `dm.loopdatasize` - Specifies the size to use when creating the loopback file for the "data" - device which is used for the thin pool. The default size is 100G. Note that - the file is sparse, so it will not initially take up this much space. + >**Note**: This option configures devicemapper loopback, which should not be used in production. + + Specifies the size to use when creating the loopback file for the + "data" device which is used for the thin pool. The default size is + 100G. The file is sparse, so it will not initially take up this + much space. Example use: @@ -212,10 +232,12 @@ Currently supported options of `devicemapper`: * `dm.loopmetadatasize` + >**Note**: This option configures devicemapper loopback, which should not be used in production. + Specifies the size to use when creating the loopback file for the - "metadata" device which is used for the thin pool. The default size is 2G. - Note that the file is sparse, so it will not initially take up this much - space. + "metadadata" device which is used for the thin pool. The default size + is 2G. The file is sparse, so it will not initially take up + this much space. Example use: @@ -248,6 +270,8 @@ Currently supported options of `devicemapper`: * `dm.datadev` + (Deprecated, use `dm.thinpooldev`) + Specifies a custom blockdevice to use for data for the thin pool. If using a block device for device mapper storage, ideally both datadev and @@ -256,12 +280,12 @@ Currently supported options of `devicemapper`: Example use: - $ docker -d \ - --storage-opt dm.datadev=/dev/sdb1 \ - --storage-opt dm.metadatadev=/dev/sdc1 + $ docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1 * `dm.metadatadev` + (Deprecated, use `dm.thinpooldev`) + Specifies a custom blockdevice to use for metadata for the thin pool. For best performance the metadata should be on a different spindle than the @@ -270,13 +294,11 @@ Currently supported options of `devicemapper`: If setting up a new metadata pool it is required to be valid. This can be achieved by zeroing the first 4k to indicate empty metadata, like this: - $ dd if=/dev/zero of=$metadata_dev bs=4096 count=1 + $ dd if=/dev/zero of=$metadata_dev bs=4096 count=1 Example use: - $ docker -d \ - --storage-opt dm.datadev=/dev/sdb1 \ - --storage-opt dm.metadatadev=/dev/sdc1 + $ docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1 * `dm.blocksize` @@ -325,7 +347,7 @@ Currently supported options of `devicemapper`: To allow the `docker` daemon to start, regardless of `udev` sync not being supported, set `dm.override_udev_sync_check` to true: - $ docker -d --storage-opt dm.override_udev_sync_check=true + $ docker -d --storage-opt dm.override_udev_sync_check=true When this value is `true`, the `devicemapper` continues and simply warns you the errors are happening. @@ -337,6 +359,7 @@ Currently supported options of `devicemapper`: > Otherwise, set this flag for migrating existing Docker daemons to > a daemon with a supported environment. + ## Docker execdriver option Currently supported options of `zfs`: diff --git a/man/docker.1.md b/man/docker.1.md index 9b9320db3a..98135e50ff 100644 --- a/man/docker.1.md +++ b/man/docker.1.md @@ -297,79 +297,198 @@ inside it) # STORAGE DRIVER OPTIONS -Options to storage backend can be specified with **--storage-opt** flags. The -only backend which currently takes options is *devicemapper*. Therefore use these +Docker uses storage backends (known as "graphdrivers" in the Docker +internals) to create writable containers from images. Many of these +backends use operating system level technologies and can be +configured. + +Specify options to the storage backend with **--storage-opt** flags. The only +backend that currently takes options is *devicemapper*. Therefore use these flags with **-s=**devicemapper. +Specifically for devicemapper, the default is a "loopback" model which +requires no pre-configuration, but is extremely inefficient. Do not +use it in production. + +To make the best use of Docker with the devicemapper backend, you must +have a recent version of LVM. Use `lvm` to create a thin pool; for +more information see `man lvmthin`. Then, use `--storage-opt +dm.thinpooldev` to tell the Docker engine to use that pool for +allocating images and container snapshots. + Here is the list of *devicemapper* options: +#### dm.thinpooldev + +Specifies a custom block storage device to use for the thin pool. + +If using a block device for device mapper storage, it is best to use +`lvm` to create and manage the thin-pool volume. This volume is then +handed to Docker to create snapshot volumes needed for images and +containers. + +Managing the thin-pool outside of Docker makes for the most feature-rich method +of having Docker utilize device mapper thin provisioning as the backing storage +for Docker's containers. The highlights of the LVM-based thin-pool management +feature include: automatic or interactive thin-pool resize support, dynamically +changing thin-pool features, automatic thinp metadata checking when lvm activates +the thin-pool, etc. + +Example use: `docker -d --storage-opt dm.thinpooldev=/dev/mapper/thin-pool` + #### dm.basesize -Specifies the size to use when creating the base device, which limits the size -of images and containers. The default value is 10G. Note, thin devices are -inherently "sparse", so a 10G device which is mostly empty doesn't use 10 GB -of space on the pool. However, the filesystem will use more space for the empty -case the larger the device is. **Warning**: This value affects the system-wide -"base" empty filesystem that may already be initialized and inherited by pulled -images. -#### dm.loopdatasize -Specifies the size to use when creating the loopback file for the "data" -device which is used for the thin pool. The default size is 100G. Note that the -file is sparse, so it will not initially take up this much space. +Specifies the size to use when creating the base device, which limits +the size of images and containers. The default value is 10G. Note, +thin devices are inherently "sparse", so a 10G device which is mostly +empty doesn't use 10 GB of space on the pool. However, the filesystem +will use more space for base images the larger the device +is. -#### dm.loopmetadatasize -Specifies the size to use when creating the loopback file for the "metadadata" -device which is used for the thin pool. The default size is 2G. Note that the -file is sparse, so it will not initially take up this much space. +This value affects the system-wide "base" empty filesystem that may already +be initialized and inherited by pulled images. Typically, a change to this +value requires additional steps to take effect: + + $ sudo service docker stop + $ sudo rm -rf /var/lib/docker + $ sudo service docker start + +Example use: `docker -d --storage-opt dm.basesize=20G` #### dm.fs -Specifies the filesystem type to use for the base device. The supported -options are "ext4" and "xfs". The default is "ext4" + +Specifies the filesystem type to use for the base device. The +supported options are `ext4` and `xfs`. The default is `ext4`. + +Example use: `docker -d --storage-opt dm.fs=xfs` #### dm.mkfsarg + Specifies extra mkfs arguments to be used when creating the base device. +Example use: `docker -d --storage-opt "dm.mkfsarg=-O ^has_journal"` + #### dm.mountopt + Specifies extra mount options used when mounting the thin devices. -#### dm.datadev -Specifies a custom blockdevice to use for data for the thin pool. +Example use: `docker -d --storage-opt dm.mountopt=nodiscard` -If using a block device for device mapper storage, ideally both datadev and -metadatadev should be specified to completely avoid using the loopback device. +#### dm.use_deferred_removal + +Enables use of deferred device removal if `libdm` and the kernel driver +support the mechanism. + +Deferred device removal means that if device is busy when devices are +being removed/deactivated, then a deferred removal is scheduled on +device. And devices automatically go away when last user of the device +exits. + +For example, when a container exits, its associated thin device is removed. If +that device has leaked into some other mount namespace and can't be removed, +the container exit still succeeds and this option causes the system to schedule +the device for deferred removal. It does not wait in a loop trying to remove a busy +device. + +Example use: `docker -d --storage-opt dm.use_deferred_removal=true` + +#### dm.loopdatasize + +**Note**: This option configures devicemapper loopback, which should not be used in production. + +Specifies the size to use when creating the loopback file for the +"data" device which is used for the thin pool. The default size is +100G. The file is sparse, so it will not initially take up +this much space. + +Example use: `docker -d --storage-opt dm.loopdatasize=200G` + +#### dm.loopmetadatasize + +**Note**: This option configures devicemapper loopback, which should not be used in production. + +Specifies the size to use when creating the loopback file for the +"metadadata" device which is used for the thin pool. The default size +is 2G. The file is sparse, so it will not initially take up +this much space. + +Example use: `docker -d --storage-opt dm.loopmetadatasize=4G` + +#### dm.datadev + +(Deprecated, use `dm.thinpooldev`) + +Specifies a custom blockdevice to use for data for a +Docker-managed thin pool. It is better to use `dm.thinpooldev` - see +the documentation for it above for discussion of the advantages. #### dm.metadatadev -Specifies a custom blockdevice to use for metadata for the thin pool. -For best performance the metadata should be on a different spindle than the -data, or even better on an SSD. +(Deprecated, use `dm.thinpooldev`) -If setting up a new metadata pool it is required to be valid. This can be -achieved by zeroing the first 4k to indicate empty metadata, like this: - - dd if=/dev/zero of=/dev/metadata_dev bs=4096 count=1 +Specifies a custom blockdevice to use for metadata for a +Docker-managed thin pool. See `dm.datadev` for why this is +deprecated. #### dm.blocksize -Specifies a custom blocksize to use for the thin pool. The default blocksize -is 64K. + +Specifies a custom blocksize to use for the thin pool. The default +blocksize is 64K. + +Example use: `docker -d --storage-opt dm.blocksize=512K` #### dm.blkdiscard -Enables or disables the use of blkdiscard when removing devicemapper devices. -This is enabled by default (only) if using loopback devices and is required to -resparsify the loopback file on image/container removal. -Disabling this on loopback can lead to *much* faster container removal times, -but will prevent the space used in `/var/lib/docker` directory from being returned to -the system for other use when containers are removed. +Enables or disables the use of `blkdiscard` when removing devicemapper +devices. This is disabled by default due to the additional latency, +but as a special case with loopback devices it will be enabled, in +order to re-sparsify the loopback file on image/container removal. -# EXAMPLES -Launching docker daemon with *devicemapper* backend with particular block devices -for data and metadata: +Disabling this on loopback can lead to *much* faster container removal +times, but it also prevents the space used in `/var/lib/docker` directory +from being returned to the system for other use when containers are +removed. - docker -d -s=devicemapper \ - --storage-opt dm.datadev=/dev/vdb \ - --storage-opt dm.metadatadev=/dev/vdc \ - --storage-opt dm.basesize=20G +Example use: `docker -d --storage-opt dm.blkdiscard=false` + +#### dm.override_udev_sync_check + +By default, the devicemapper backend attempts to synchronize with the +`udev` device manager for the Linux kernel. This option allows +disabling that synchronization, to continue even though the +configuration may be buggy. + +To view the `udev` sync support of a Docker daemon that is using the +`devicemapper` driver, run: + + $ docker info + [...] + Udev Sync Supported: true + [...] + +When `udev` sync support is `true`, then `devicemapper` and `udev` can +coordinate the activation and deactivation of devices for containers. + +When `udev` sync support is `false`, a race condition occurs between +the`devicemapper` and `udev` during create and cleanup. The race +condition results in errors and failures. (For information on these +failures, see +[docker#4036](https://github.com/docker/docker/issues/4036)) + +To allow the `docker` daemon to start, regardless of whether `udev` sync is +`false`, set `dm.override_udev_sync_check` to true: + + $ docker -d --storage-opt dm.override_udev_sync_check=true + +When this value is `true`, the driver continues and simply warns you +the errors are happening. + +**Note**: The ideal is to pursue a `docker` daemon and environment +that does support synchronizing with `udev`. For further discussion on +this topic, see +[docker#4036](https://github.com/docker/docker/issues/4036). +Otherwise, set this flag for migrating existing Docker daemons to a +daemon with a supported environment. # EXEC DRIVER OPTIONS