Last week, Docker Inc’s reorganization where it sold its Docker Enterprise business to Mirantis, and received $35 million in fresh VC funding to focus on developer tooling shook the container and virtualization industry. Loyal users of Docker Swarm, the orchestration solution pushed by the company that kicked off the container revolution, are picking up the pieces and asking about the future of SwarmKit, the open source project behind Swarm.
Has Kubernetes won? Is Docker Swarm reaching end of life in 2 years? No, and no. Based on good authority from prominent Docker Captains and the lead maintainer of Swarm, Drew Erny, Swarm is definitely not dead. Here’s our take on what Docker Captains Bret Fisher (@bretfisher), Jeff Nickoloff (@allingeek), Michael Irwin (@mikesir87), and Nirmal Mehta (@normalfaults) had to say about the future of Docker.
Compared to Kubernetes, Swarm has always been the “go to” for small teams adopting container infrastructure for the first time – because of it’s ease of use and tight integration with Docker itself. It’s built into the Docker Engine and you can get a Swarm cluster going with one, simple command. The Swarm CLI is a natural extension of the Docker commands that everybody knows and loves. This can’t be said for Kubernetes, especially if you set it up the “hard way” instead of going for a managed solution from one of the major cloud providers. Following Docker’s “batteries included, but replaceable” philosophy, Swarm handles consensus between manager nodes, and mesh networking between swarm nodes, right out of the box.
Right out of the gate, here are some myths about Swarm that the Captains debunked about Swarm during their Youtube Live.
MYTH: Swarm is only for small companies. – 80% of the Fortune 150 companies using Docker Enterprise (the commercial version of Docker) began their containerization journey with Swarm, prior to when Docker EE began supporting Kubernetes. At the time of the sale to Mirantis, Docker’s enterprise business had over 750 enterprise customers, including household names like Intuit, Liberty Mutual, PayPal, Visa, and more. While some (most?) of the new customers of Docker EE are probably coming onboard with Kubernetes, many mega-corporations still rely on Swarm for their container orchestration needs.
“Swarm does not scale well” is a lazy argument made by some proponents of Kubernetes. It’s also an argument that is clearly disproved by the roster of customers who use Swarm at “world scale”, who have extensively shared their customer stories. If you’ve filed a tax return using TurboTax or swiped your Visa credit card, there were containers orchestrated by Swarm behind the scenes which made it happen.
MYTH: Swarm is “getting the ax.” Mirantis promises at least two years of support for Swarm and SwarmKit. Major Swarm users on Docker EE will continue to generate revenue for Mirantis, and expect continued support for the investment in Swarm.
The lead maintainer of SwarmKit, Drew Erny, has been acqui-hired by Mirantis to continue working on Swarm. The bug fixes will continue to be upstreamed to the open source project, and while it’s unclear how many new features Mirantis will invest in, Erny hopes to work on a Container Storage Interface (CSI) for Swarm, which will make it easier to write plugins to handle persistent volumes in a Swarm container cluster – on a wider range of backend storage options.
MYTH: Swarm is not (no longer) open source. Mirantis only purchased the intellectual property for the closed source components of Docker, including Docker Enterprise Engine, Docker Trusted Registry, Docker Unified Control Plane, and the Docker Enterprise CLI. The open source components of Docker, including SwarmKit, remain with Docker Inc. and continue to be open source. If you currently use the open source version of Docker Swarm, there will be no change in the functionality that is available to you. The proprietary version of Swarm with some paid features found in Docker EE, will continue to be updated and maintained by some 300 Docker employees who moved to Mirantis. Although the new Docker Inc’s stated focus is going back to its roots to work on developer tooling such as Docker Hub, Docker Desktop, Docker, and Docker Compose CLI, it’s likely that Swarm will be considered as part of “developer tooling” and the remaining employees at Docker will continue making commits to SwarmKit. Also, many of the Moby open source projects are no longer reliant on maintainers who are on Docker Inc’s payroll.
MYTH: Kubernetes has won and Swarm is irrelevant. The reality is there is probably room for both Swarm and Kubernetes (perhaps plus a yet uninvented orchestration tool) to thrive in the enormous containerized infrastructure market. Bret Fisher noted that in recent user surveys, Swarm has not drastically declined in popularity versus Kubernetes, while Nikoloff points out that many new users of Kubernetes could be unaccounted for because they are using managed Kubernetes solutions “under the hood.” Regardless, Swarm still ships with every copy of Docker Desktop and the community Docker Engine, meaning that a entire generation of developers who are new to Docker and containers will still be exposed to Swarm. Fisher mentions that a pain point around using both Swarm and Kubernetes in an organization are the different YAML formats used to define stacks and services – Docker Compose files vs. Service Definitions. It is possible that the Docker team could develop an interpreter that translates Kubernetes service definitions into a format that can be consumed by Docker’s own tooling.
Although Kubernetes is a remarkable work of engineering contributed to open source by Google and other CNCF members, the difficulty of standing up and maintaining a cluster, and UX challenges affecting admin/operator experience continue to put Kubernetes out of reach of smaller teams without the engineering resources of Amazon, Microsoft, and Google.
For this reason, tools such as the Portainer UI, aimed at smaller shops who prioritize ease of operation for their container infrastructure have always focused on interoperating with Swarm rather than Kubernetes. In the future, Portainer plans to add support for deploying containers and application stacks to managed Kubernetes services from its graphical interface, but in a simplified and opinionated way.
Most observers in the community are optimistic about the reorganized Docker Inc’s future, since the company has a fresh start with $35M to get to revenue with its remaining 100 or so employees. Providing a developer-friendly UX has always been one of Docker’s strengths, and there’s no reason to suspect that a renewed focus on providing an application packaging standard for the “build-and-ship pipeline” can only be a good thing for the company. Although Docker has went through a lot of upheaval where almost all of the original engineers who worked on Docker’s core runtimes have moved on, the new CEO, Scott Johnston, seems well qualified to take the helm. He has been in various executive positions with the company since 2014, most recently as the Chief Product Officer, where he has witnessed the changing of the guard between three CEOs from Ben Golub, to Steve Singh, and now, himself.
Although Solomon Hykes has not announced plans to return to Docker Inc. in any capacity, many loyal fans of Docker hope that Hykes may one day come back to play an instrumental role in the container company, like Steve Jobs did to revive Apple’s fortunes in 1997.
The Autoize team is excited about the bright future that the new Docker Inc. has ahead, and for the reasons outlined in this article, we continue to recommend Swarm as a open source container orchestration solution to deploy containerized apps in any cloud-based environment.