DevOps champion Chef's recipe for collaboration

Profile picture for user pwainewright By Phil Wainewright April 23, 2017
Agile and DevOps have put developers at the forefront of collaborative teamwork in the digital enterprise. So what tools do they use? I ask IT toolmaker Chef

Team of ants carrying pebbles © juanjo -
The emergence of agile methodologies and DevOps teams have put application developers at the forefront of new, highly collaborative styles of working in the digital enterprise. Working in small, self-directed, dynamic, cross-functional teams, these organizations are at the forefront of building highly responsive, adaptable systems and processes that enable the enterprise to keep pace with the fast-moving demands of today's digitally adept customers.

So what collaboration tools do they rely on? As pioneers of digitally enabled patterns of teamwork that are now spreading beyond the IT function into other parts of the enterprise, maybe others can learn from their experience.

That's why I took the opportunity recently to catch up with Julian Dunn, Product Manager at DevOps toolmaker Chef, to ask how the company collaborates. Chef provides IT automation tools to power the continuous delivery of new code that's at the heart of the DevOps model. It has grown up alongside the DevOps movement, and its use of collaboration tools has evolved throughout that time.

As is common in the agile development world, several of the tools it uses to help manage development, such as Jira, Confluence and Trello, are from IT teamwork tools vendor Atlassian, who arranged for me to speak to Dunn. Chef also uses less specialized tools for more informal instant conversations, such as Slack for discussions and Zoom for video chat.

While the Atlassian collaboration tools are more familiar to developers, their use at Chef has spread outside of the IT function. That may become more prevalent elsewhere as DevOps practices expand into business functions such as product management and customer engagement.

Lean development processes

One of the most important tools at Chef is the Jira ticketing and helpdesk system, which has been in use for almost a decade. Over that time, says Dunn, it's evolved to become "much more of a foundation for lean development processes," incorporating concepts such as Kanban, a technique borrowed from just-in-time manufacturing to manage the flow of work through a development team. Because it breaks down work into atomic tasks, Kanban is better suited to a continuous delivery approach than the scrum methodology where developers work in fixed-length sprints, typically of two weeks, to deliver a set of tasks at the end of the sprint.

In the early days, Jira was simply used for ticketing, and there were even three completely separate implementations, says Dunn. But this meant that a backlog of work items built up without any context about their business value. This is a common problem in some agile environments, he explains.

Sometimes you get this 'agile backlog of sadness.' Especially scrum, where you have these backlog refinement teams where you sit down and go through lists of hundreds of items to work on. But without the business value attached, it's impossible to know which ones to prioritize.

Organized around outcomes

The work tickets that Chef creates in Jira today are effectively Kanban cards that define the tasks each team will work on. But they're no longer created ahead of time, to avoid building up a backlog of unused items. Instead, the business context is defined first so that when a work item is created, the team understands where it fits into the bigger picture.

Our utilization of Jira has modelled and improved as we've improved our business processes.

Jira is where we write the actual user stories. It really is about the context. We see Jira as part of a holistic product development system.

The overall product strategy is discussed and planned in a separate product management tool, and then the tickets are created in Jira as needed to manage the flow of work through development. Teams are organized by outcome. For example, product leadership may define a six-month goal to create a new product. Then that overall goal is broken down into smaller steps, says Dunn.

We then break that down into: mission statement, who are the customers, what are the outcome metrics, what are the next tactical steps a team is going to work on? — and engineers are allocated to that.

The next few steps are up to the individual teams, which are structured as a triad that combines product management, software engineering and UX design. Team members may vary during the life of the project, depending on what skills are needed along the way, says Dunn.

Collaboration across timezones

Often a team will not all be based in the same location, which increases the importance of the Jira system to communicate context, Dunn explains.

Chef is a very distributed company. Many of our engineers work from different locations — we're looking for the best talent. There's no real opportunity for them to physically sit together. We rely on communications tools and having really good engineering processes.

Some teams cross timezones, with team members in Europe and the US. Where possible the US participants in such teams are in Eastern or Central, but that's not always possible and a team may stretch from Central European to Pacific timezones. Some team members compensate by adjusting their working hours. There are also specified handoffs at certain times of the day.

For more immediate communications, people use Slack for messaging and, when a face-to-face conversation becomes essential, Zoom video chat.

That's how we compensate for not being able to walk over to somebody's desk. Rule of thumb for Slack is, if the conversation's gone on for ten minutes or more, it's time to get on a video call.

Spreading into other functions

In addition to these collaboration tools, Chef has been using Confluence for almost as long as Jira to capture engineering knowledge. As the Atlassian product has evolved, its use within Chef has spread out into other functions, says Dunn.

We not only have engineering to capture processes but also our success operation, legal, IT operations, helpdesk, and the hosted service operations team. It's very much a knowledge base for us.

Trello, which Atlassian acquired at the beginning of this year, is another product with a long history at Chef, and is also used in various functions, as Dunn explains:

We've been Trello users for many years. It's a very lightweight way for people that aren't necessary technical to get what they need outside of Jira. Legal, HR uses it. Our employee handbook is a Trello board. Marketing collaborate with our PR folk.

It's really more of a Kanban thing — it's a really easy way for folks who are business oriented to get those kinds of benefit without having to learn a developer tool.

Marketing, meet technology

But in product management, those who are able to grasp the technology make the most valuable contribution, says Dunn.

One of the interesting threads is the rise in the role of modern product management and what that means. Product management 2.0 for very digitally connected and sophisticated companies can be a technical role, in terms of technical depth rather than marketing backgrounds.

How does context get maintained and how does the business understand what technology can bring them? It's about making sure engineering has the right business context, so they can make the right decision without having these traditional requirements documents spelled out.

We have marketing people that come to it from a technical background. It gives them the right context to bring to the table. Product management people that have come from a technology background — those are unicorns.

My take

One of the fascinating aspects of agile development is the way that collaboration becomes intrinsic to the way teams are structured and dictates how work is organized. This feels different from other spheres, where collaboration has a supporting role and is added on as a supplementary layer.

A Kanban approach in particular forces transparency and collaboration because each individual work item forms part of a larger project and therefore must be visible to others who have an interest in its progress.

It's interesting to see how Chef has harnessed several different tools that each make their own contribution to collaboration within a highly devolved and distributed team structure.