How to define a disk quota for docker containers?

Question

I'm was faced with the requirement to have disk quotas on docker containers. Specifically I want to limit the amount of data that is not in the layers of base image but in the diff. Googling for "docker disk quota" suggests to use either the device mapper or the btrfs backends. While being able to have quotas in both backends (with different semantics) both have their issues:

• btrfs isn't stable enough for production use
• device mapper only limits the complete container size including the base image but doesn't allow limiting the diff

What is the best way to solve this?

Headwater

Answer 1

One way to solve this is to put the diff directory /var/lib/docker/aufs/diff/$CONTAINER_ID into a sparse loopback mounted ext4 directory. This effectively limits the amount of data a user can store/modify in a container. This is the bash code I use:

do_enable_quota() {
    local ID=$1
    local QUOTA_MB=$2

    local LOOPBACK=/var/lib/docker/aufs/diff/$ID-loopback
    local LOOPBACK_MOUNT=/var/lib/docker/aufs/diff/$ID-loopback-mount
    local DIFF=/var/lib/docker/aufs/diff/$ID

    docker stop -t=0 $ID
    sudo dd of=$LOOPBACK bs=1M seek=$QUOTA_MB count=0
    sudo mkfs.ext4 -F $LOOPBACK
    sudo mkdir -p $LOOPBACK_MOUNT
    sudo mount -t ext4 -n -o loop,rw $LOOPBACK $LOOPBACK_MOUNT
    sudo rsync -rtv $DIFF/ $LOOPBACK_MOUNT/
    sudo rm -rf $DIFF
    sudo mkdir -p $DIFF
    sudo umount $LOOPBACK_MOUNT
    sudo rm -rf $LOOPBACK_MOUNT
    sudo mount -t ext4 -n -o loop,rw $LOOPBACK $DIFF
    docker start $ID    
}

This approach works perfectly for me but the drawback is that I need to wrap the "start", "stop" and "rm" commands to take the mount into account.

Headwater

would you have an example how to get this script to work. In my case, only want to limit the ubuntu:14:04 container size to a few MiB, and have the container created. – Iselaisenberg

I get these error after running cmd: $ do_enable_quota 91e54dfb11794fad694460162bf0cb0a4fa710cfa3f60979c177d920813e267c 20 – Iselaisenberg

sent 177 bytes received 50 bytes 454.00 bytes/sec total size is 0 speedup is 0.00 Error response from daemon: no such id: 91e54dfb11794fad694460162bf0cb0a4fa710cfa3f60979c177d920813e267c Error: failed to start containers: [91e54dfb11794fad694460162bf0cb0a4fa710cfa3f60979c177d920813e267c] – Iselaisenberg

have you completed working approach? Container specific disk quota, may be limiting inodes. – Kathrinekathryn

Answer 2

ZFS is also a great deal: https://docs.docker.com/engine/userguide/storagedriver/zfs-driver/

This way you can manage your disk pools with 'zpool' command line.

For example, to create 'just a bunch of vdisk':

[root@localhost /]# mkdir /dsk
[root@localhost /]# dd if=/dev/zero of=/dsk/disk1 bs=1M count=750
[root@localhost /]# dd if=/dev/zero of=/dsk/disk2 bs=1M count=750
[root@localhost /]# dd if=/dev/zero of=/dsk/disk3 bs=1M count=750
[root@localhost /]# dd if=/dev/zero of=/dsk/disk4 bs=1M count=750

Ok, we have four disks of 750M each one. Now create one single ZFS pool:

[root@localhost /]# zpool create CIALINUX /dsk/disk{1,2,3,4}

Now we have a volume named CIALINUX, with almost 3GB automatically mounted in our / directory:

[root@localhost /]# df -h |grep CIALINUX

CIALINUX 2,9G 18K 2,9G 1% /CIALINUX

Other commands you can explore by yourself are:

# zpool list
# zpool status
# zpool status -x
# zpool destroy CIALINUX

Important: This last one 'destroys' your pool. Please pay attention to it.

Once you have your zfs pools, just place your files attached as docker volumes inside /zpool-mounted-directory for each container.

Hope this help community.

Gastroscope

Note this is not a kind of software RAID, you have no four disks writing distributed I/O so your performance will be not so rich as a RAID volume. – Gastroscope