This is part two of our series on using GitLab and Rancher together to build a CI/CD pipeline, and follows partone from last week, which covered deploying, configuring, and securing GitLab in Rancher. We’ve also madethe entire walkthrough available for download.
Using GitLab CI Multi-Runner to Build Containers
GitLab CI is a powerful tool for continuous integration and continuous delivery. To use it with Rancher, we’ll deploy a runner that will execute jobs.
Launching the Runner
There are several ways that runners can be deployed, but since we’ll be targeting building containers from our repositories, we’ll run a Docker container that has direct access to /var/run/docker.sock to build images that are siblings to itself.
In Rancher, add a service to your Gitlab stack
Set it up with the following configuration:
When the container launches, it will create a default configuration in /etc/gitlab-runner, to which we’ve connected a volume. The next step is to register the runner with your Gitlab instance.
The options that I’m setting below are correct for a basic runner that will build any job. You can also limit runners to specific repositories or use other images. Read the documentation from Gitlab to learn what options are best for your environment.
One of the things that often surprises administrators when they first begin working with Docker containers is the fact that containers natively use non-persistent storage. When a container is removed, so too is the container’s storage.
Of course, containerized applications would be of very limited use if there were no way of enabling persistent storage. Fortunately, there are ways to implement persistent storage in a containerized environment. Although a container’s own native storage is non-persistent, a container can be connected to storage that is external to the container. This allows for the storage of persistent data, since this external storage is not removed when a container is stopped.
The first step in deciding how to go about implementing persistent storage for your containers is to determine the underlying type of storage system that you will use. In this regard, there are three main options that are generally available: File system storage, block storage, and object storage. Below, I explain the differences between each type of storage and what they mean when it comes to setting up storage for a containerized environment.
Each time a new software technology arrives on the scene, InfoSec teams can get a little anxious. And why shouldn’t they? Their job is to assess and mitigate risk – and new software introduces unknown variables that equate to additional risk for the enterprise. It’s a tough job to make judgments about new, evolving, and complex technologies; that these teams approach unknown, new technologies with skepticism should be appreciated.
This article is an appeal to the InfoSec people of the world to be optimistic when it comes to containers, as containers come with some inherent security advantages:
Containers may be super cool, but at the end of the day, they’re just another kind of infrastructure. A seasoned developer is probably already familiar with several other kinds of infrastructure and approaches to deploying applications. Another is not really that big of a deal.
However, when the infrastructure creates new possibilities with the way an application is architected—as containers do—that’s a huge deal. That is why the services in a microservice application are far more important than the containerized infrastructure they run on.
Modularity has always been a goal of application architectures, and now that the concept of microservices is possible, how you build those services end up dictating where they run and how they are deployed. Services are where application functionality meets the user, and where the value your application can provide is realized.
That’s why if you want to make the most of containers, you should be thinking about more than just containers. You have to focus on services, because they’re the really cool thing that containers enable.
Services v. Containers
For conversation’s sake, using services and containers interchangeably is fine—because the ideal use case for a containerized application is one that is deconstructed into services, where each service is deployed as a container (or containers).
However, the tactics cannot be synonymous. Services are an implied infrastructure, but more importantly, an application architecture. When you talk about a service that is part of your application, that service is persistent. You can’t suddenly have an application without a login page or a shopping cart, for example, and expect things to go well.
Containers, on the other hand, are designed to live and die in very short time frames. Ideally, with every deployment or revert, the container is killed as soon as the new deployment is live and the traffic is routed to it. So containers are not persistent. And if the delivery chain is working correctly, that should not matter at all.
Microservices, as both an application and an infrastructure term, does have some unique elements associated with it, which diverge the relationship even further. Read more
So you’ve decided to use microservices. To help implement them, you may have already started refactoring your app. Or perhaps refactoring is still on your to-do list.
In either case, if this is your first major experience with refactoring, at some point, you and your team will come face-to-face with the very large and very obvious question: How do you refactor an app for microservices? That’s the question we’ll be considering in this post.
Before discussing the how part of refactoring into microservices, it is important to step back and take a closer look at the what and when of microservices. There are two overall points that can have a major impact on any microservice refactoring strategy. Read more
Since Docker launched in 2013, it has brought a level of excitement and innovation to software development that’s contagious. It has rallied support from every corner—enterprises to startups, developers to IT folk, plus the open source community, ISVs, the biggest public cloud vendors, and every tool across the software stack. Since the launch of Docker, many major milestones have served to advance the container revolution. Let’s look at some of them.
Container Orchestration Options
Getting started with your first container is fairly simple. All it takes is your laptop and a Docker client. However, running a microservices app is a whole other beast. The most difficult part is in creating, managing, and automating clusters of ephemeral containers.
The first major tool to address this challenge was Mesos with its Marathon orchestrator. Having powered distributed infrastructure even before Docker, Marathon is in use in production workloads at Twitter, and in other large-scale web applications.
The next orchestration tool to gain prominence was Kubernetes. In fact, today, Kubernetes leads the pack of Docker orchestration tools because of how extensible it is. It supports a broad list of programming languages, infrastructure options, and enjoys tremendous support from the container ecosystem. It isolates the application layer from the infrastructure layer, thus enabling true portability across multiple cloud vendors, and infrastructure setups. Read more