Why the AI industry urgently needs more uncomfortable conversations about BAME representation


Bias in Artificial Intelligence (AI) and in all forms of autonomous and automated systems is a serious issue, but not just because of the obvious risk of automating socio-economic problems, racial discrimination, gender imbalances, and postcode lotteries (which can be proxies for profiling in open data). Automating them also creates a veneer of digital neutrality, pushing the underlying problems further and further away from transparency, audit, and explainability – and therefore from critical legal concepts such as liability.

Bias may sometimes exist in AI systems’ training data. For example, a sentencing advice algorithm might be biased if it has been trained in a legal system that has a history of treating minorities less fairly than the majority. Likewise an image database that is largely of white, male faces might be used to train a facial recognition system. The result? Automated technology that may not recognise women or ethnic minorities as human. 

Bias may also be couched in the design of algorithms, in decision-making processes – or in an organisation’s assumptions about the world, which are always the first things to be automated. This was demonstrated by the recent fiasco over the UK’s A’Level results predictions. The Prime Minister labelled the system a “rogue algorithm”, as though it had somehow become sentient and acted independently. In fact, its failure simply reflected flaws in human decision-making – errors that the government was happy to stand by until the criticism grew too loud.

But most of all, bias exists in teams, a problem that affects the entire IT sector. Such bias is unlikely to be overt or conscious – no one is accusing coders of being racists, misogynists, or homophobes en masse. This deep-set ‘bias’ is really an expression of the lack of diversity in an industry that is overwhelmingly young, male, straight, and white, which may find its way into the products they create. Why? Because they are working in closed groups, designing – and often training – systems with no external reference points. 

In recent years, some white males have come to believe that they are being unfairly attacked – even discriminated against – when these issues are discussed. But when over 85% of people across all types of STEM careers are male and over 90% of them are white, that complaint doesn’t stand much scrutiny. It’s not about blame, it’s about an imbalance that needs urgent redress, because lack of diversity makes it harder to deliver technologies that work equally well for everyone.

In the UK, a number of the most senior policy-makers in AI, robotics, and related fields are women, such as Sana Khareghani, new head of Whitehall’s Office for AI, or Professor Dame Wendy Hall, co-author of the government’s AI review. But what about the vital presence of black and minority ethnic communities, which are among the least represented in the IT industry, especially in AI and coding? 

Raising black voices

Mark Martin is a computer science teacher and founder of UK Black Tech, one of several organizations that are seeking to rebalance the industry so the digital world better reflects the diversity of the real one. Others include Afrotech Fest and Coding Black Females. Martin says:

If you don't have that representation solving the biggest problems, how can we say that we're the most innovative or the most productive [nation]? The UK has a great opportunity to be the most innovative, but in order to do that we need to bake diversity and inclusion in, and also homegrown talent.

He confirms that failing to do this may have consequences in the technology itself, perpetuating problems in a world it is designed to make better:

If you don't have that representation baked within AI, one day you might come to harm, exclude, or even kill individuals that it is supposed to serve. We can't wait for these incidents to happen, in the same way as passport scanners not picking up darker skin or facial recognition systems letting through particular groups of people and not others.

Stark facts and figures

It is not as if there are too few people from minorities to work in the sector, especially in cities such as London, epicentre of the UK’s AI industry:

When you look at London in particular, it is a diverse community with nearly 40% BAME [black, asian and minority ethnic] communities. And then you look at the universities where some universities have 60% BAME populations, but under 10% overall are getting into STEM fields. That's a problem. 

In order to raise that awareness, we need to hold the system to account. There was a report published about six years ago saying that over 16% of black computer scientists are unemployed. They're sitting unemployed because recruiters are saying ‘You don't have the experience we need, we're not looking for junior hires’. There are a lot of excuses and barriers for why this talent can't enter the sector. And when we look at organizations in terms of diversity and how many have black and ethnic minorities sitting on their leadership teams, it's actually zero percent [in the UK overall].

The research cited by Martin is discussed here and analysed in this Guardian article. It suggests that overall unemployment in computer science stands at 14%. This figure is high because of much greater unemployment among black and minority ethnic people – 56% more of whom study computer science than is the average for other university courses. The conclusion: lots of black people want to work in IT, but the doors are not always open to them. This brings down the entire industry’s employment statistics at a time when many digital jobs are unfilled. 

Difficult to talk about

Martin acknowledges that these conversations make some people uncomfortable, but says: 

If we don't have the uncomfortable conversations, the vacuum gets wider at both ends. With a lot of the black professionals that I work with in the tech sector, some of them feel uncomfortable talking about racism or biases within technology. And a lot of the white people that we work with also feel uncomfortable, because they don't know the right words to say and they don't want to be deemed racist. So the two sides don't talk, they don't come together to resolve these issues.

He adds:

It's not that the talent doesn't exist. It's the access and opportunities that are hard to reach and also the awareness. A lot of these conversations are happening in silos, and then you find that certain people or groups are excluded from the conversations, because they can’t find an entry point, an access point to actually chime into the discussion.

As noted, the likes of UK Black Tech and Afrotech Fest are among the organizations seeking to provide these entry points. They do so by engaging with communities, training and up-skilling workers, starting conversations, and creating innovation hubs, work experience opportunities, and networking forums. One priority is inspiring young black people – girls as well as boys – to work in STEM careers. I asked Martin how he and his team can do that. He said:

I think they are really inspired already. I think it's about how we get the opportunity to actually get into this sector. It’s about coming in, helping them, and propping them up. How do we think about a long-term roadmap? We host mock technical interviews... it’s about giving them sustainable work experience, so they can build up their professional portfolios to get that job. We can go way beyond short-term measures into the long term to really increase that engagement.

It’s also about getting them to recognise that everything they do on a daily basis is underpinned by some type of technology. We need to have a greater conversation about belonging, because a lot of these conversations are so top level that young people feel they don't belong in these sectors.  One of the things that we've had to do is break down some of those barriers of belonging. We can say, ‘If you like protecting your computer, then one day you can do this full time in cyber-security’. Or ,‘If you like to be a force for good in your communities, you can actually get a career in this.’ 

One of the biggest things that I'm working on at the moment is how to get more young people to work within their local communities, because at the moment there's a disconnect. What we teach in education is sometimes different to what's happening in their communities. There are so many issues that we can get young people to be inspired about.

My take

Uncomfortable conversations maybe, but essential.