We benchmark, you score productivity, they surveil - the good, bad and ugly of teamwork analytics

Phil Wainewright Profile picture for user pwainewright November 27, 2020 Audio mode
Microsoft has conjured up the ugly face of teamwork analytics with its Productivity Score but the data can also be used in a good way

Graphic of hand holding magnifying glass over people © princerko - Fotolia.com
(© princerko - Fotolia.com)

It turns out that analyzing teamwork data is subject to one of those irregular verbs celebrated in the classic TV sitcom on the workings of government, Yes Minister. When we collect data to learn how to achieve more with less stress, it's a great idea. If you monitor performance to measure workplace productivity, that might be OK, depending on your motives. But when Microsoft helps employers track what their workers are up to, it's a dystopian nightmare of intrusive workplace surveillance.

A new feature in Microsoft 365 that quietly went live earlier this month has aroused a storm of indignation this week. Productivity Score is a set of analytics dashboards and reports in the Microsoft 365 Admin Center that shows aggregate metrics on how teams are using the various components of Microsoft's productivity and teamwork suite. As Jared Spataro, Corporate Vice President for Microsoft 365, explains in a blog post introducing the new feature:

Productivity Score leverages the depth and breadth of Microsoft 365 to give you visibility into how your organization works, insights to identify where you can make improvements, and actions you can take to update skills and systems so that everyone can do their best work. It shows you people experiences across five categories: content collaboration, meetings, communication, teamwork, and mobility. Meanwhile, technology experiences focus on endpoints, network connectivity, and Microsoft 365 Apps.

So far, so innocuous. Trouble is, managers can also dig down into the behavior of individual employees, for example to check up on whether they're sharing cloud files instead of emailing attachments, or how much time they're spending in Teams instead of their Outlook inbox. Austrian researcher Wolfie Christl, author of the book Networks of Control on corporate surveillance, was first to highlight the scope for individual surveillance in a thread on Twitter:

Christl also notes that Microsoft has already been offering a tool called Workplace Analytics that collects similar data, and that both tools connect into the Microsoft Graph, a database of the connections and interactions people make at work. To that extent, the monitoring provided in Productivity Score is nothing new — it's just been packaged up for managers in a new way that's turned spectacularly ugly in Microsoft's hands.

The good side to teamwork analytics

The trouble with stirring up so much furore is that it creates the impression that any collection and monitoring of data on how people work in teams is a bad thing. Yet as anyone who's owned a personal fitness monitor realizes, digitally monitoring your behavior is the first step to identifying how to improve your results. The same applies to teamwork, and the maturity model we've mapped out for digital teamwork at diginomica emphasizes the contribution that intelligent measurement and data science should play at the highest levels. Collecting and analyzing teamwork data can be a good thing. What's important is to collect and manage that data in a way that respects the participants' rights to privacy and self-actualization.

Talking to some of the other vendors in the space reveals more examples of the good side to teamwork analytics. Alex Hood, Head of Product at the work management tool Asana, says his product emphasizes accountability for completion of tasks rather than time spent using specific tools. Transparency about who's done what, and how it builds up to the overall goals, is particularly important when team members are working remotely from each other or in different time zones. He says it helps people see how their work contributes to the whole and is part of the dynamics of successful distributed teamwork.

At Workfront, collecting data as part of the automation of teamwork processes "lays the foundation" for more sophisticated automations later on to help manage workloads, says Darin Patterson, Director, Product Management. One customer who presented at the company's recent analyst day spoke about its use of Workfront data to avoid overcommitting resources or setting unrealistic deadlines.

Even Microsoft is capable of conceiving of ways to use teamwork analytics for the wellbeing of team members. One of the earliest mentions of Productivity Score was during the Ignite conference in September, couched in the context of monitoring people's wellbeing in the midst of the pandemic. Obviously someone realized there were other aspects of the feature that might go down less well.

The full horror of Productivity Score is that it breaks pretty much all the rules of how such data should be used. There's no transparency over what data is being collected and how it's being modeled, access to the analysis is limited to managers so that individuals can't see what data is being collected about them, and consent is assumed rather than explicitly sought. Its release has been an object lesson in how not to introduce this kind of capability.

My take

The ease with which data can be collected and analyzed as workplaces become more and more digital opens up many issues of trust, transparency and responsibility. The technology is far in advance of the ethical framework in which it should be operating, and this week's debate around the Productivity Score has exposed many of the issues that need addressing.

There is a strong case for analyzing teamwork patterns to find opportunities to improve performance, but it has to be done in a way that recognizes people are not machines. The danger is that such metrics will collect insufficient data, apply simplistic models, and end up imposing unreasonable demands on workers that will ultimately produce disaffection and burnout.

A pattern of behavior that works well for one person may not be optimal for their colleague. The whole point of teamwork is to allow each person to contribute what they do best, and the most effective teams prize diverse approaches. A system that is able to analyze all of these inputs and help each team member work out their own route to achieving more with less stress would be a good outcome. Using digital surveillance to impose Tayloristic performance targets is its ugly flipside.

UPDATE December 1st - In a blog post today, Jared Spataro, Corporate Vice President for Microsoft 365, has announced that Microsoft is removing user names from Productivity Score and making it clear the product is "a measure of organizational adoption of technology — and not individual user behavior." He concludes: "We appreciate the feedback we’ve heard over the last few days and are moving quickly to respond by removing user names entirely from the product. This change will ensure that Productivity Score can’t be used to monitor individual employees."

A grey colored placeholder image