mirror of
https://github.com/moby/moby.git
synced 2022-11-09 12:21:53 -05:00
d715803d45
Commit 09ee269d ("devmapper: Add option for specifying the thin pool
blocksize") also switched the default dm-thin-pool blocksize from 64K to
512K. That change unfortunately breaks the activation of dm-thin-pool
devices that were previously created using a 64K blocksize. Here is an
example of the dm-thin-pool activation failure users may experience:
device-mapper: thin: 253:4: pool target (204800 blocks) too small: expected 1638400
device-mapper: table: 253:4: thin-pool: preresume failed, error = -22
The reason for this is docker is passing 512K as the blocksize for a
dm-thin-pool that was previously created using a 64K blocksize. Docker
doesn't record the blocksize the is used when it creates a dm-thin-pool.
Until now it never had a need to do so because the blocksize was always
hardcoded. The dm-thin-pool blocksize must be the same every time a
dm-thin-pool is activated.
As a stop-gap fix, revert to using 64K for the default blocksize.
But we do need a proper fix for this now that 'dm.blocksize' is exposed
as a proper storage option. One possible fix would be to record the
blocksize for each dm-thin-pool that docker creates and to pass that
recorded blocksize down in the dmsetup table load each time the
dm-thin-pool is activated (this would be comparable to what lvm2 does).
Docker-DCO-1.1-Signed-off-by: Mike Snitzer <snitzer@redhat.com> (github: snitm)
152 lines
4.9 KiB
Markdown
152 lines
4.9 KiB
Markdown
## devicemapper - a storage backend based on Device Mapper
|
|
|
|
### Theory of operation
|
|
|
|
The device mapper graphdriver uses the device mapper thin provisioning
|
|
module (dm-thinp) to implement CoW snapshots. For each devicemapper
|
|
graph location (typically `/var/lib/docker/devicemapper`, $graph below)
|
|
a thin pool is created based on two block devices, one for data and
|
|
one for metadata. By default these block devices are created
|
|
automatically by using loopback mounts of automatically creates sparse
|
|
files.
|
|
|
|
The default loopback files used are `$graph/devicemapper/data` and
|
|
`$graph/devicemapper/metadata`. Additional metadata required to map
|
|
from docker entities to the corresponding devicemapper volumes is
|
|
stored in the `$graph/devicemapper/json` file (encoded as Json).
|
|
|
|
In order to support multiple devicemapper graphs on a system the thin
|
|
pool will be named something like: `docker-0:33-19478248-pool`, where
|
|
the `0:33` part is the minor/major device nr and `19478248` is the
|
|
inode number of the $graph directory.
|
|
|
|
On the thin pool docker automatically creates a base thin device,
|
|
called something like `docker-0:33-19478248-base` of a fixed
|
|
size. This is automatically formated on creation and contains just an
|
|
empty filesystem. This device is the base of all docker images and
|
|
containers. All base images are snapshots of this device and those
|
|
images are then in turn used as snapshots for other images and
|
|
eventually containers.
|
|
|
|
### options
|
|
|
|
The devicemapper backend supports some options that you can specify
|
|
when starting the docker daemon using the --storage-opt flags.
|
|
This uses the `dm` prefix and would be used somthing like `docker -d --storage-opt dm.foo=bar`.
|
|
|
|
Here is the list of supported options:
|
|
|
|
* `dm.basesize`
|
|
|
|
Specifies the size to use when creating the base device, which
|
|
limits the size of images and containers. The default value is
|
|
10G. Note, thin devices are inherently "sparse", so a 10G device
|
|
which is mostly empty doesn't use 10 GB of space on the
|
|
pool. However, the filesystem will use more space for the empty
|
|
case the larger the device is.
|
|
|
|
Example use:
|
|
|
|
``docker -d --storage-opt dm.basesize=20G``
|
|
|
|
* `dm.loopdatasize`
|
|
|
|
Specifies the size to use when creating the loopback file for the
|
|
"data" device which is used for the thin pool. The default size is
|
|
100G. Note that the file is sparse, so it will not initially take
|
|
up this much space.
|
|
|
|
Example use:
|
|
|
|
``docker -d --storage-opt dm.loopdatasize=200G``
|
|
|
|
* `dm.loopmetadatasize`
|
|
|
|
Specifies the size to use when creating the loopback file for the
|
|
"metadadata" device which is used for the thin pool. The default size is
|
|
2G. Note that the file is sparse, so it will not initially take
|
|
up this much space.
|
|
|
|
Example use:
|
|
|
|
``docker -d --storage-opt dm.loopmetadatasize=4G``
|
|
|
|
* `dm.fs`
|
|
|
|
Specifies the filesystem type to use for the base device. The supported
|
|
options are "ext4" and "xfs". The default is "ext4"
|
|
|
|
Example use:
|
|
|
|
``docker -d --storage-opt dm.fs=xfs``
|
|
|
|
* `dm.mkfsarg`
|
|
|
|
Specifies extra mkfs arguments to be used when creating the base device.
|
|
|
|
Example use:
|
|
|
|
``docker -d --storage-opt "dm.mkfsarg=-O ^has_journal"``
|
|
|
|
* `dm.mountopt`
|
|
|
|
Specifies extra mount options used when mounting the thin devices.
|
|
|
|
Example use:
|
|
|
|
``docker -d --storage-opt dm.mountopt=nodiscard``
|
|
|
|
* `dm.datadev`
|
|
|
|
Specifies a custom blockdevice to use for data for the thin pool.
|
|
|
|
If using a block device for device mapper storage, ideally both
|
|
datadev and metadatadev should be specified to completely avoid
|
|
using the loopback device.
|
|
|
|
Example use:
|
|
|
|
``docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1``
|
|
|
|
* `dm.metadatadev`
|
|
|
|
Specifies a custom blockdevice to use for metadata for the thin
|
|
pool.
|
|
|
|
For best performance the metadata should be on a different spindle
|
|
than the data, or even better on an SSD.
|
|
|
|
If setting up a new metadata pool it is required to be valid. This
|
|
can be achieved by zeroing the first 4k to indicate empty
|
|
metadata, like this:
|
|
|
|
``dd if=/dev/zero of=$metadata_dev bs=4096 count=1```
|
|
|
|
Example use:
|
|
|
|
``docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1``
|
|
|
|
* `dm.blocksize`
|
|
|
|
Specifies a custom blocksize to use for the thin pool. The default
|
|
blocksize is 64K.
|
|
|
|
Example use:
|
|
|
|
``docker -d --storage-opt dm.blocksize=512K``
|
|
|
|
* `dm.blkdiscard`
|
|
|
|
Enables or disables the use of blkdiscard when removing
|
|
devicemapper devices. This is enabled by default (only) if using
|
|
loopback devices and is required to res-parsify the loopback file
|
|
on image/container removal.
|
|
|
|
Disabling this on loopback can lead to *much* faster container
|
|
removal times, but will make the space used in /var/lib/docker
|
|
directory not be returned to the system for other use when
|
|
containers are removed.
|
|
|
|
Example use:
|
|
|
|
``docker -d --storage-opt dm.blkdiscard=false``
|