The four secrets of building a great DevOps team

Profile picture for user Dominic Wellington By Dominic Wellington April 9, 2020
Summary:
MongoDB Director, Dominic Wellington, asks the question - what are you doing to make your DevOps team truly great?

Image of MongoDB’s Dominic Wellington Working from home
(Image sourced via MongoDB)

Lately I've been on lots of Zoom calls — perhaps you have too? As we've rolled into the second month of lockdown (for some of us) those calls now regularly turn to what's next and the bigger picture: how we will work and what our teams will look like in the future?

We all want to build, and be part of, teams that deliver excellent software. We want a growth culture and to help our business reach its objectives. But how do we get there?

For those without kids at home, or who have managed to distract the kids for a moment, one silver lining of lockdown life is reading, and recently I got to re-read two excellent pieces of research which give us some clues.

The first was the State of DevOps report from Google Cloud. It's now a few months old, but remains a worthwhile read.

The second was a study from MongoDB that calls for security in DevOps to overcome competing priorities in organisations.

Reading through the two pieces, there were some very useful ideas for anyone who wants to create a great DevOps team that will thrive in the next decade.

You should read both in full, but in case you can’t, I pulled out the four of the most useful ideas. I suspect the pandemic may open up more flexible working structures for many people and organisations, but many of the fundamental elements of what makes a great team will not change. So here’s what the research tells us about those fundamentals: 

1) Understand what an elite team looks like

We like to think we’re part of the elite team. Unfortunately, in the same way that 50% of people think they’re above average drivers, it can’t possibly be true. To understand what is needed to become elite, we'd have to define what “elite” even means.

In their State of DevOps report, the Google authors use two models and a lot of numbers to measure what good really looks like. It's worth wrapping our head around it.

First, and perhaps most importantly, the report looks at software delivery and operational (SDO) performance. There were five measures which made up a score for this category. The metrics were:

  1. Deployment frequency
  2. Lead time for changes
  3. Time to restore service
  4. Change failure rate
  5. Availability

Image of MongoDB DevOps performance metrics
(Image sourced via MongoDB)

The good news is these are not mutually exclusive. From the report:

Many professionals approach these metrics as representing a set of trade-offs, believing that increasing throughput will negatively impact the reliability of the software delivery process and the availability of services. For six years in a row, however, our research has consistently shown that speed and stability are outcomes that enable each other. 

Participant teams in the research were grouped along with these metrics and the difference between the best and the worst was startling. Lead time in elite teams is 106x faster and they deploy 208x more frequently. It makes sense: practice makes perfect, and if change is a routine activity, the team will get very very good at managing change.

This is the peak that we’re all aiming for.

Image charting delivery of software delivery performance
(Image sourced via MongoDB)

But being elite is not just an abstract academic ideal. It matters.

The analysis shows elite performers are twice as likely to meet or exceed their organisational performance goals.

Not tech goals, we're talking company-wide imperatives. Like increasing customer numbers, profitability, customer satisfaction, quality products or operating efficiency. Now you care. Now your CEO cares.

2) Prioritise developer experience

It’s hard to define, it’s hard to measure, but my god it matters. Developer productivity is ensuring developers have the right tools in the right context is an essential element of an elite team.

The report goes into some detail on all of the technical aspects you might expect: cloud, using as-a-service platforms for undifferentiated tasks, technical capabilities, paying down technical debt and many others. Again there was a high correlation between things like full cloud adoption and a successful DevOps team.

However, what I found most fascinating was the other piece of the productivity equation. Despite occasional evidence to the contrary, developers are human. Building a great team is much more than picking the right technology and smartest people. As you can see in the image, the report argues that technology is an important part, but only one part, of building a really productive team. The possibly-mythical “10x engineer” rarely builds a good team around themselves, let alone an elite one. Eliteness is what happens when everybody has the opportunity to be great.

Image mapping productivity in a DevOps team
(Image sourced via MongoDB)

3. Ensure psychological safety

A culture of psychological safety was highly correlated with high-performance teams.

From the report:

Our analysis found that this culture of psychological safety is predictive of software delivery performance, organizational performance, and productivity. Although Project Aristotle measured different outcomes, these results indicate that teams with a culture of trust and psychological safety, with meaningful work and clarity.

The best part of combining good technology choices with psychological safety was it created a virtuous cycle. This meant developers were able to leave at a reasonable hour and not burn out. In turn that encouraged better productivity.

It's all connected. The report claimed to revalidated previous years’ research and found that good technical practices and improved processes can reduce employee burnout.

4) Give it some sec – DevSecOps

Turning to the recent security research, MongoDB's Joe Drumgoole looked at the heart of a crucial piece of developing great technology teams: responsibility.

Who is responsible for what in a DevOps world?

When asked, ‘who has the most responsibility for securing an application?’, just  29% of developers cited themselves. The rest pointed to others, such as security specialists (22%), the business leaders who briefed the project (18%), the ops team (16%) and even security members they don’t know (14%).

This is not necessarily a problem – as long as those teams work together in a joined-up process. Developers focus on building functionality, security teams specialise in making that functionality safe. As long as everyone has a common goal and talks to each other, different specialists can work together very effectively to build as a team.

Image charting who in an organisation holds responsibility for securing applications
(Image sourced via MongoDB)

There can be a tension between developer experience, fast delivery and security but those teams that understand their priorities and work in a psychologically safe environment can find the balance. If you move too fast and miss an element, be it security or culture, that is a debt that will need to be paid at some point. Security is an aspect of functionality; if you don’t build it in, you’ll have to hack it together in a hurry later, and then you’ll be stuck dealing with the resulting shortcuts and compromises.

Now build it

The speed, agility and reliability of IT drive the whole organisation. Elite DevOps teams are elite in the sense that they can deliver what the rest of the organisation needs, and do it quickly and reliably. Adding Security into the mix is just the next step in that process, ensuring that the systems and processes the organisation relies on will keep working for them – and not embarrass them in public.

If you can build an elite DevSecOps team, it may well be your company's unfair advantage over the next decade.  

What I'm taking from these two reports is that the real challenge for leaders is not necessarily the technology. It's ensuring we can prioritise developer experience in all its forms, give our teams psychological safety and develop clear lines of responsibility.

So, what are you doing to make your team of technologists truly great?