DevOps is dead! Long live platform engineering!
Here we go again: another technology hype cycle of The New Big Thing, and The Old Big Thing is Dead. But as someone who still believes in DevOps (despite seeing many sub-optimal initiatives) And As someone who really believes in modern platform engineering, I’d like to delve into this topic with a bit more detail and a history of my time in this space. All histories are incomplete, but some are useful.
A Brief History of Platform Engineering
Creating digital platforms as a way to deliver software at scale is not a recent invention, and predates the emergence of the DevOps movement in the late 2000s. Many big tech companies whose primary business was building software realized decades ago that they could integrate developer teams more quickly and with higher quality through standardization of infrastructure, building self-service interfaces, increasingly high-level abstractions Enables you to build, ship, and operate applications by providing On dedicating a team to solve developer problems and maintaining all of them as a single platform.
However, these teams had to build and operate those platforms completely from scratch, which required a pool of technically sophisticated (and well-compensated) engineers with operational skills, executive support, and organizational focus. , and who were relatively unaffected with legacy IT. At least compared to your average enterprise company. The tools we take for granted these days around infrastructure are because code didn’t exist and as an industry, we hadn’t yet experienced the transformative impact of public cloud platforms.
In the early 2000s, Amazon created a shared internal IT platform that was described as “undifferentiated heavy-lifting” so that developers could better focus on shipping value to customers. A few years later, it became available to users outside of Amazon, and quickly evolved from a web application platform to an infrastructure-as-a-service offering that changed our entire industry. Around 2003 or 2004, Google created a dedicated SRE organization and began work on Borg, but the company did not publicly publicize these initiatives until it published a whitepaper on Borg in 2015 and a year or two Haven’t published a book on SRE since.
The cross-pollination of employees in modern big tech companies meant that these ideas and attitudes began to spread – because they worked.
Beginning of DevOps
The inefficiencies in many large enterprise development organizations and the problems faced by development teams by increasingly complex infrastructure had been apparent for some time. Attempts to solve this have included “Golden Image” virtual machines, self-service software catalogs, and the first few attempts at PaaS, often with a good level of success for new greenfield applications, but with much less success for legacy and commercial applications. off-the-shelf applications. Implementing the mandate in a diverse internal landscape (or at least, more diverse than that of Big Tech companies) was a significant challenge, especially given the regulatory burden under which many of them operated with relatively heavy and decades of manual processes. .
Parallel to most of these, but in a somewhat different environment, the whole DevOps movement began in the late 2000s, with one of the key starting moments being John Allspaugh and Paul Hammond’s 2009 VelocityConf talk “10+ Deploys Per Day: Dev and Ops Collaboration on Flickr” in which he stressed the importance of aligning communication, collaboration, and incentives between operations and development. An entire community emerged to investigate and pursue these ideas, and interesting open source projects have exploded. .
Much of this work took place in the open, and the ideas were adopted by a wide variety of organizations. DevOps was influenced by and borrowed from a number of prior movements and frameworks, including Agile, Lean Manufacturing, The Toyota Way, and concepts from psychology, cognitive science, organizational dynamics, and industrial security. This is why I’ve always thought that DevOps is best defined as a loose collection of practices and processes developed that take context into account.
Scaling DevOps Is Difficult
As we’ve tracked the State of DevOps Report Puppet for several years, while many companies have been successful in implementing DevOps principles and practices, a significant proportion of enterprises are “stuck in the middle”, with good success achieved by individual teams. level, but not consistently throughout the organization.
In 2018, we recognized for the first time that DevOps success within the enterprise requires significant standardization on how self-service is delivered as part of our five-stage evolutionary model.
DevOps evolutionary model
Rise of Platform Engineering
Platform engineering is nothing new, but it’s not a particularly accessible concept if you haven’t experienced it yourself. In 2019, some major analyst firms began to recognize this as a trend, and Manuel Pace and Matthew Skelton published Team Topology – an in-depth examination of the topic based on their extensive experience doing IT consulting and observing patterns. practice. He not only outlined the organizational structure but also provided directional advice on organizational dynamics – which, in my experience, is a major stumbling block for more traditional organizations.
Platform engineering and DevOps align
Given how many large companies have struggled to experience the benefits of DevOps in their organizations, and that this more prescriptive movement of modern platform engineering is increasingly proving to deliver value, some have argued That “DevOps is dead” and that modern platform engineering has supplanted it.
This is not true, and we do ourselves a disservice as an industry if we perpetuate it. DevOps has always borrowed ideas from other movements, and platform engineering is just another one to add to the list for organizations of a certain scale and complexity. If you’re working in a small company with a handful of developers (some who are more inclined to the operational side than others), you may have no need to take a platform approach, and the principles of DevOps still apply. There is a great guiding function.
DevOps is about using automation as a tool to align incentives and increase collaboration across all teams involved in the software delivery lifecycle to deliver software better, more quickly, and with less stress . Modern platform engineering has adopted the pre-existing platform approach and has a clear focus on treating the platform as a Product rather as a Assignment or ProjectPlus clear guidance for where teams should interact through collaboration, and where they should interact through self-service interfaces.
Like DevOps, platform engineering makes heavy use of automation, focuses on collaboration, requires empathy across organizational functions, and puts people rather than technology front and center. It aligns perfectly with DevOps, and is proving to be a viable way for many enterprises to do DevOps at scale in highly complex and diverse environments.
DevOps is not dead. It is currently under development.
Get the full details on how platform engineering enables DevOps at scale in Puppet’s upcoming DevOps report: Platform Engineering Edition. Sign up now to receive the report.