d715803d45
Commit 09ee269d ("devmapper: Add option for specifying the thin pool
blocksize") also switched the default dm-thin-pool blocksize from 64K to
512K. That change unfortunately breaks the activation of dm-thin-pool
devices that were previously created using a 64K blocksize. Here is an
example of the dm-thin-pool activation failure users may experience:
device-mapper: thin: 253:4: pool target (204800 blocks) too small: expected 1638400
device-mapper: table: 253:4: thin-pool: preresume failed, error = -22
The reason for this is docker is passing 512K as the blocksize for a
dm-thin-pool that was previously created using a 64K blocksize. Docker
doesn't record the blocksize the is used when it creates a dm-thin-pool.
Until now it never had a need to do so because the blocksize was always
hardcoded. The dm-thin-pool blocksize must be the same every time a
dm-thin-pool is activated.
As a stop-gap fix, revert to using 64K for the default blocksize.
But we do need a proper fix for this now that 'dm.blocksize' is exposed
as a proper storage option. One possible fix would be to record the
blocksize for each dm-thin-pool that docker creates and to pass that
recorded blocksize down in the dmsetup table load each time the
dm-thin-pool is activated (this would be comparable to what lvm2 does).
Docker-DCO-1.1-Signed-off-by: Mike Snitzer <snitzer@redhat.com> (github: snitm)
4.9 KiB
devicemapper - a storage backend based on Device Mapper
Theory of operation
The device mapper graphdriver uses the device mapper thin provisioning
module (dm-thinp) to implement CoW snapshots. For each devicemapper
graph location (typically /var/lib/docker/devicemapper, $graph below)
a thin pool is created based on two block devices, one for data and
one for metadata. By default these block devices are created
automatically by using loopback mounts of automatically creates sparse
files.
The default loopback files used are $graph/devicemapper/data and
$graph/devicemapper/metadata. Additional metadata required to map
from docker entities to the corresponding devicemapper volumes is
stored in the $graph/devicemapper/json file (encoded as Json).
In order to support multiple devicemapper graphs on a system the thin
pool will be named something like: docker-0:33-19478248-pool, where
the 0:33 part is the minor/major device nr and 19478248 is the
inode number of the $graph directory.
On the thin pool docker automatically creates a base thin device,
called something like docker-0:33-19478248-base of a fixed
size. This is automatically formated on creation and contains just an
empty filesystem. This device is the base of all docker images and
containers. All base images are snapshots of this device and those
images are then in turn used as snapshots for other images and
eventually containers.
options
The devicemapper backend supports some options that you can specify
when starting the docker daemon using the --storage-opt flags.
This uses the dm prefix and would be used somthing like docker -d --storage-opt dm.foo=bar.
Here is the list of supported options:
-
dm.basesizeSpecifies the size to use when creating the base device, which limits the size of images and containers. The default value is 10G. Note, thin devices are inherently "sparse", so a 10G device which is mostly empty doesn't use 10 GB of space on the pool. However, the filesystem will use more space for the empty case the larger the device is.
Example use:
docker -d --storage-opt dm.basesize=20G -
dm.loopdatasizeSpecifies the size to use when creating the loopback file for the "data" device which is used for the thin pool. The default size is 100G. Note that the file is sparse, so it will not initially take up this much space.
Example use:
docker -d --storage-opt dm.loopdatasize=200G -
dm.loopmetadatasizeSpecifies the size to use when creating the loopback file for the "metadadata" device which is used for the thin pool. The default size is 2G. Note that the file is sparse, so it will not initially take up this much space.
Example use:
docker -d --storage-opt dm.loopmetadatasize=4G -
dm.fsSpecifies the filesystem type to use for the base device. The supported options are "ext4" and "xfs". The default is "ext4"
Example use:
docker -d --storage-opt dm.fs=xfs -
dm.mkfsargSpecifies extra mkfs arguments to be used when creating the base device.
Example use:
docker -d --storage-opt "dm.mkfsarg=-O ^has_journal" -
dm.mountoptSpecifies extra mount options used when mounting the thin devices.
Example use:
docker -d --storage-opt dm.mountopt=nodiscard -
dm.datadevSpecifies a custom blockdevice to use for data for the thin pool.
If using a block device for device mapper storage, ideally both datadev and metadatadev should be specified to completely avoid using the loopback device.
Example use:
docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1 -
dm.metadatadevSpecifies a custom blockdevice to use for metadata for the thin pool.
For best performance the metadata should be on a different spindle than the data, or even better on an SSD.
If setting up a new metadata pool it is required to be valid. This can be achieved by zeroing the first 4k to indicate empty metadata, like this:
``dd if=/dev/zero of=$metadata_dev bs=4096 count=1```
Example use:
docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1 -
dm.blocksizeSpecifies a custom blocksize to use for the thin pool. The default blocksize is 64K.
Example use:
docker -d --storage-opt dm.blocksize=512K -
dm.blkdiscardEnables or disables the use of blkdiscard when removing devicemapper devices. This is enabled by default (only) if using loopback devices and is required to res-parsify the loopback file on image/container removal.
Disabling this on loopback can lead to much faster container removal times, but will make the space used in /var/lib/docker directory not be returned to the system for other use when containers are removed.
Example use:
docker -d --storage-opt dm.blkdiscard=false