On Monday, the guys at CoreOS announced Rocket, a command-line tool for working with App Container, the company's own container image format, runtime and discovery mechanism. It was the first major competitive blow levied against Docker. The news within the news was that Rocket’s format and runtime promises to be completely open, which is in contrast to the approach Docker has taken, having shown consternation around publishing or agreeing on a spec /standard around their container technology.
Docker co-founder and CTO, Solomon Hykes, responded with a Tweetstorm, ending with this tweet:
Docker, Inc. finds itself in the always thorny position of a company balancing its responsibilities as the steward of an open source project and as a profit-making entity. Invariably as Docker guides the community towards its one version of a universal truth and builds out a fully-featured enterprise management stack, some devs will get left behind – that’s simply the cost of doing business for a company behind an open source project as opposed to one that’s commercializing a foundation-led project, say like Cloudera has done with Hadoop or Red Hat with Linux. Clearly, CoreOS saw an opportunity to pounce, and I’m sure we’ll see more vendors come to market with competing container technologies.
If we’re to chart out how this evolves, the closest analog is the virtualization market. VMware was the clear winner as it was first to successfully commercialize its hypervisor technology, taking the ESX – which actually had its roots in open source – and building a full management stack around it which became vSphere. VMware came to dominate the market with Xen (Citrix), KVM and Microsoft picking up relative scraps.
The hypervisor helped commoditize the OS and physical infrastructure, giving way to the cloud-based architectures we have today. Similarly now, we’re in the midst of another platform shift. Docker offers a higher level of abstraction, taking the OS, virtual machine, physical machine and infrastructure provider, and commoditizing it all. The difference this time around is that in concert with a platform shift (VMs to containers), we're also seeing an architectural shift in the way applications are built (evolving from monoliths to systems that are distributed, highly available and modular). Martin Fowler provides a fantastic overview of distributed systems and micro-services architecture here.
The thing is, building, running and scaling distributed applications is HARD and wholly requires a re-thinking of the tools and systems we’ve had in the place for the last decade, with underlying container technology serving as the modular component for others to build on. We are now re-answering questions like how do you manage networking, persistence, storage in these highly distributed environments? And hence why we’re seeing the emergence of new platforms and tools like Mesos, Kubernetes, etcd, Weave, Flocker, etc.
So what’s really at stake isn’t just domination in one portion of the stack – the underlying container technology – but rather a whole a new stack in and of itself: an integrated offering to manage all aspects of running and building containers in a cluster. And that’s why announcements like yesterday’s really really matter.