At the DockerCon conference today in Austin, Docker announced a new service called the Modernize Traditional Applications (MTA) Program that enables customers to move certain legacy apps into Docker containers, put them under management of Docker Enterprise Edition and prepare them for use on more modern infrastructure. What’s more, the company is so confident in their ability to move… Read More
Mirantis, one of the earliest players in the OpenStack ecosystem, today announced that it will end-of-life Mirantis OpenStack support in September 2019. The Mirantis Cloud Platform, which combines OpenStack with the Kubernetes container platform (or which could even be used to run Kubernetes separately), is going to take its place. While Mirantis is obviously not getting out of the OpenStack… Read More
While CoreOS is probably still best known for its Linux distribution, that was only the company’s gateway drug to a wider range of services. Tectonic, the company’s service for running Kubernetes-based container deployments in the enterprise, now sits at the core of its business. Until now, Tectonic could only be used for installing and managing Kubernetes on bare-metal and… Read More
When it comes to implementing a system like Docker’s container platform, it normally takes a very particular set of technical skills. Docker wants to remove some of the complexity involved in running their product set, and to that end released Docker Enterprise Edition today. Think of the enterprise edition as a package of tools designed to work seamlessly across any supported flavor… Read More
A few years ago, using containers to run your applications came with a number of security tradeoffs compared to using virtual machines. As enterprises started adopting various container technologies faster than anybody expected, that became an issue and companies like Docker started making security a priority. For Docker especially, that work is starting to pay off. The company today… Read More
The Cloud Native Computing Foundation (CNCF), the open source group that’s also the home of the popular Kubernetes container management system, today announced that it has added Linkerd as its fifth hosted project. Linkerd (pronounced linker-DEE), which was incubated at Buoyant, follows in the footsteps of other CNCF projects like Kubernetes, Prometheus, OpenTracing and Fluentd. The… Read More
With the launch of Windows Server 2016 three months ago, Microsoft gave its users the ability to use the Docker engine to run containers on Windows server. This meant developers could now package their Windows executables into containers and run them on Windows Server (though obviously not on Linux machines), using the same Docker engine and commands they were already used to. Today, AWS… Read More
Docker announced today that it was open sourcing containerd (pronounced Container D), making a key infrastructure piece of its container platform available for anyone to work on.
Containerd, which acts as the core container runtime engine, is a component within Docker that provides “users with an open, stable and extensible base for building non-Docker products and container… Read More
For years, the public face of Kubernetes was one of the project’s founders: Google group product manager Craig McLuckie. He started the open-source container-management project together with Joe Beda, Brendan Burns and a few other engineers inside of Google, which has since brought it under the guidance of the newly formed Cloud Native Computing Foundation.
Beda became an… Read More
This blog post reviews the appropriateness of Docker and other container solutions for your database environment.
A few weeks back, I wrote a fairly high-level blog post about containers. It covered what you should consider when thinking about using Docker, rkt, LXC, etc. I hope you’ve taken the chance to give it a quick read. It’s a good way to understand some of the disciplines you need to consider before moving to a new technology. However, it sparked a conversation in our Solutions Engineering team. Hopefully, the same one that you’re having in your organization: should customers run their database in containers?
Before we start, I’ll admit that Percona uses containers. Percona Monitoring and Management (PMM for short) presents all of the pretty graphs and query analytics by running in a Docker container. We made that choice because the integration between the components is where we could provide the most value to users. Docker lets us distribute a single ready-to-go unit of awesomeness. In short, it has huge potential on the application side of your environment.
However, for databases… here are some of our recommendations:
Quick n Dirty
Decision = NOT FOR DBs (as it sits right now)
This is not the case for every environment. It is the default that we think is the best recommendation for the majority of our customers. Please note, that I am only making this recommendation for your database. If you’re using microservices for your application today, then it could make more sense to containerize your database depending on the load characteristics of your database, your scaling needs and the skillset you currently have.
Lack of Synergy
Before you decide to shoot me, please take some time to understand where we’re coming from. First of all, people designed container solutions to deal with stateless applications that have ephemeral data. Containers spin up a quick microservice and then destroy it. This includes all the components of that container (including its cache and data). The transient nature of containers is because all of the components and services of that container are considered to be part of the container (essentially it’s all or nothing). Serving the container a data volume owned by the underlying OS by punching a hole through the container can be very challenging. Current methods are too unreliable for most databases.
Most of the development efforts put into the various solutions had one goal in mind: statelessness. There are solutions that can help keep your data persistent, but they are very quickly evolving. From what we can tell, they require a high level of complexity, that negate any efficiency gains due to increased operational complexity (and risk). To further my point, this is precisely the conclusion that we’ve come to time and again when we’ve reviewed any “real world” information about the use of containers (especially Docker).
They’re Just Not Stable Yet
These container solutions are meant for quick development and deployment of applications that are broken into tiny components: microservices. Normally, these applications evolve very quickly in organizations that are very software/developer driven. That seems to be how these container solutions (again, especially Docker) are developed as well. New features are pushed out with little testing and design. The main focus seems to be the latest featureset and being first to market. They “beg for forgiveness” instead of “ask for permission.” On top of that, backward compatibility (from what we can tell) is a distant concern (and even that might be an overstatement). This means that you’re going to have to have a mature Continuous Delivery and testing environment as well as a known and tested image repository for your containers.
These are awesome tools to have for the right use cases, but they take time, money, resources and experience. In speaking with many of our customers, this is just not where they’re at as an organization. Their businesses aren’t designed around software development, and they simply don’t have the checkbooks to support the resources needed to keep this hungry machine fed. Rather, they are looking for something stable and performant that can keep their users happy 24×7. I know that we can give them a performant, highly-available environment requires much less management if we strip out containers.
Is There Hope?
Absolutely, in fact, there’s a lot more than hope. There are companies running containers (including databases) at massive scale today! These are the types of companies that have very mature processes. Their software development is a core part of their business plan and value proposition. You probably know who I’m talking about: Uber, Google, Facebook (there are more, these are a just a few). There’s even a good rundown of how you can get persistence in containers from Joyent. But as I said before, the complexity needed to get the basic features necessary to keep your data alive and available (the most basic use of a database) is much too high. When containers have a better and more stable solution for persistent storage volumes, they will be one step closer to being ready, in my opinion. Even then, containerizing databases in most organizations that aren’t dealing with large scale deployments (50+ nodes) with wildly varying workloads is probably unnecessary.
Don’t’ Leave Us Hanging…
I realize that the statement “you’re probably not ready to containerize your database” does not constitute a solution. So here it is: the Solutions Engineering team (SolEng for short) has you covered. Dimitri Vanoverbeke is in the process of a great blog series on configuration management. Configuration management solutions can greatly increase the repeatability of your infrastructure, and make sure that your IT/App Dev processes are repeatable in the physical configuration of your environment. Automating this process can lead to great gains. However, this should make use of a mature development/testing process as part of your application development lifecycle. The marriage of process and technology creates stable applications and happy customers.
Besides configuration management as an enhanced solution, there are some services that can make the life of your operations team much easier. Service discovery and health checking come to mind. My favorite solution is Consul, which we use extensively in PMM for configuration and service metadata. Consul can make sure that your frontend applications and backend infrastructure are working from a real-time snapshot of the state of your services.
There is a lot to think about when it comes to managing an environment, especially when your application develops at a quick pace. With the crafty use of available solutions, you can reduce the overhead that goes into every release. On top of that, you can increase resiliency and availability. If you need our help, please reach out. We’d love to help you!