Even with the almost unimaginable efficiencies achieved by the major public cloud providers, at any given time they still have excess capacity that is left idle. And as an incentive to try to get some return on those resources, both AWS and Google Compute Engine are willing to sell those resources at a steep discount, usually starting at around 90%.
What’s the catch? Well, the prices are market driven, set by the highest bidder. It’s a classic marketplace model: demand drives the value of assets. The challenge for public cloud users, however, is that at any given time the spot instance you are using can be reclaimed if someone outbids you. With Amazon, you have two minutes to vacate the instance before it is terminated; Google Cloud gives you 30 seconds.
This volatility has kept the bulk of companies using public clouds away from this model. How can I expect to keep my application running if I can lose a server at any given moment, especially if setting up the server to be production ready takes significant amount of time? It is not uncommon for configuration management tools to take 10 or more minutes to install packages and deploy an application. The time it takes to set up a server and the narrow window of time to vacate makes it extremely challenging to make effective use of these discounted instance types.
How containers help optimize cloud costs
Well as you might have guessed, containers can help with this obstacle to using the spot market. The pre-built nature of containers means startup times can be drastically smaller than with a dynamic, scripted or configuration management-driven approach. The required packages, application code, and various files have all been figured out at build time, and written to essentially a compressed archive (Docker image). This means startup times for your application in the sub-minute time frame are now within reach. Read more
One of the more novel concepts in systems design lately has been the notion of serverless architectures. It is no doubt a bit of hyperbole as there are certainly servers involved, but it does mean we get to think about servers differently.
The potential upside of serverless
Imagine a simple web based application that handles requests from HTTP clients. Instead of having some number of program runtimes waiting for a request to arrive, then invoking a function to handle them, what if we could start the runtime on-demand for each function as a needed and throw it away afterwards? We wouldn’t need to worry about the number of servers running that can accept connections, or deal with complex configuration management systems to build new instances of your application when you scale. Additionally, we’d reduce the chances of common issues with state management such as memory leaks, segmentation faults, etc.
Perhaps most importantly, this on-demand approach to function calls would allow us to scale every function to match the number of requests and process them in parallel. Every “customer” would get a dedicated process to handle their request, and the number of processes would only be limited by the compute capacity at your disposal. When coupled with a large cloud provider whose available, on-demand compute sufficiently exceeds your usage, serverless has the potential to remove a lot of the complexity around scaling your application.
The cloud vs. on-premises debate is an old one. It goes back to the days when the cloud was new and people were trying to decide whether to keep workloads in on-premises datacenters or migrate to cloud hosts.
But the Docker revolution has introduced a new dimension to the debate. As more and more organizations adopt containers, they are now asking themselves whether the best place to host containers is on-premises or in the cloud.
As you might imagine, there’s no single answer that fits everyone. In this post, we’ll consider the pros and cons of both cloud and on-premises container deployment and consider which factors can make one option or the other the right choice for your organization. Read more