zfs service will install the kernel-headers for your kernel (if you build your own kernel, you’ll need to replicate this service), and then download the ZFS on Linux source, and build and install it. Then it will build a
zfs-tools image that will be used to give you access to the zfs tools.
The only restriction is that you must mount your zpool into
/mnt, as this is the only shared mount directory that will be accessible throughout the system-docker managed containers (including the console).
$ sudo ros service enable zfs
$ sudo ros service up zfs
# you can follow the progress of the build by running the following command in another ssh session:
$ sudo ros service logs --follow zfs
# wait until the build is finished.
$ lsmod | grep zfs
Note: if you switch consoles, you may need to re-run
ros up zfs.
After it’s installed, it should be ready to use. Make a zpool named
zpool1 using a device that you haven’t yet partitioned (you can use
sudo fdisk -l to list all the disks and their partitions).
Note: You need to mount the zpool in
/mntto make it available to your host and in containers.
$ sudo zpool list
$ sudo zpool create zpool1 -m /mnt/zpool1 /dev/<some-disk-dev>
$ sudo zpool list
$ sudo zfs list
$ sudo cp /etc/* /mnt/zpool1
$ docker run --rm -it -v /mnt/zpool1/:/data alpine ls -la /data
To experiment with ZFS, you can create zpool backed by just ordinary files, not necessarily real block devices. In fact, you can mix storage devices in your ZFS pools; it’s perfectly fine to create a zpool backed by real devices and ordinary files.
zdb command may be used to display information about ZFS pools useful to diagnose failures and gather statistics. By default the utility tries to load pool configurations from
/etc/zfs/zpool.cache. Since the RancherOS ZFS service does not make use of the ZFS cache file and instead detects pools by inspecting devices, the
zdb utility has to be invoked with the
E.g. to show the configuration for the pool
zpool1 you may run the following command:
$ sudo zdb -e -C zpool1
First, you need to stop the
docker system service and wipe out
$ sudo system-docker stop docker
$ sudo rm -rf /var/lib/docker/*
To enable ZFS as the storage driver for Docker, you’ll need to create a ZFS filesystem for Docker and make sure it’s mounted.
$ sudo zfs create zpool1/docker
$ sudo zfs list -o name,mountpoint,mounted
At this point you’ll have a ZFS filesystem created and mounted at
/zpool1/docker. According to Docker ZFS storage docs, if the Docker root dir is a ZFS filesystem, the Docker daemon will automatically use
zfs as its storage driver.
Now you’ll need to remove
-s overlay (or any other storage driver) from the Docker daemon args to allow docker to automatically detect
$ sudo ros config set rancher.docker.storage_driver 'zfs'
$ sudo ros config set rancher.docker.graph /mnt/zpool1/docker
# Now that you've changed the Docker daemon args, you'll need to start Docker
$ sudo system-docker start docker
After customizing the Docker daemon arguments and restarting
docker system service, ZFS will be used as Docker storage driver:
$ docker info
Server Version: 1.12.6
Storage Driver: zfs
Zpool: error while getting pool information strconv.ParseUint: parsing "": invalid syntax
Zpool Health: not available
Parent Dataset: zpool1/docker
Space Used By Parent: 19456
Space Available: 8256371200
Parent Quota: no
Logging Driver: json-file
Cgroup Driver: cgroupfs
Network: host bridge null overlay
Default Runtime: runc
Security Options: seccomp
Kernel Version: 4.9.6-rancher
Operating System: RancherOS v0.8.0-rc8
Total Memory: 1.953 GiB
Docker Root Dir: /mnt/zpool1/docker
Debug Mode (client): false
Debug Mode (server): false