If you have been working with Docker for any length of time, you probably already know that shared volumes and data access across hosts is a tough problem. While the Docker ecosystem is maturing, implementing persistent storage across environments still seems to be a problem for most folks. Luckily, Rancher has been working on this problem and come up with a unique solution that addresses most of these issues. Running a database with shared storage still isn’t widely recommended, but for many other use cases, sharing volumes across hosts is good practice.
Much of the guide was inspired by one of the Rancher Online meetups. Additionally, here is a little reference to go from that includes some of the NFS configuration information if you want to build something like this yourself from scratch.
If you haven’t heard of it yet, the Convoy project by Rancher is aimed at making persistent volume storage easy. Convoy is a very appealing volume plugin because it offers a variety of different options. For example, there is EBS volume and S3 support, along with VFS/NFS support, giving users some great and flexible options for provisioning shared storage.
This is a little recipe for standing up a Dockerized NFS server for the convoy-nfs service to connect to. Docker-NFS is basically a poor man’s EFS, and you should only run this if you are confident that the server won’t get destroyed or the data simply isn’t important enough to matter if it is lost. You can find more information about the Docker NFS server I used here.
Today, our team at Rancher announced an exciting new feature called Persistent Storage Services. Persistent storage support builds on the work we’ve done with Rancher Convoy, and makes it dramatically easier to run stateful applications in production using Rancher. The Docker volume plugins, introduced in Docker 1.8 and further enhanced in Docker 1.9, enables developers to utilize a variety of persistent storage implementations as standard Docker volumes. Our new Persistent Storage Services capability complements Docker volume plugins by providing a backend implementation of a Docker volume plugin, and is the core storage technology in our recently announced hyper-converged infrastructure stack for Docker.
Rancher’s new Persistent Storage Services have four primary characteristics:
A persistent storage service is a software-defined storage system deployed as containers. It typically includes multiple containers running on a cluster of hosts to provide redundancy and failover, building a distributed, resilient storage system out of local disk volumes.
A persistent storage service is deployed in conjunction with one or more applications that utilize it as persistent storage.
Rancher deploys and manages persistent storage services. Rancher configures the required Docker volume plugin to make persistent storage services consumable as Docker volumes. In addition, Rancher provides the service discovery and cross-app linking capabilities to enable the application to call any API exposed by the persistent storage service.
Applications can create Docker volumes backed by a persistent storage service, and can also go through service discovery to invoke vendor-specific APIs implemented by the persistent storage service.