AI development under current gendered lens is bad---leads to reinvestment into patriarchy

Artificial intelligence () is set to change the world. It’ll upend society and reshape the way we work and interact. But how it’ll impact humanity depends on your gender.

AI researchers are warning that the current revolution in technology could disproportionately impact women more negatively than men because of a raft of different reasons, from who is employed by the big tech companies leading the way and the datasets the technology is trained on.

“It’s a reinforcement of a technocentric patriarchal society that once again encodes gender bias into seemingly ‘neutral’ technology that will become ubiquitous and melded into our lives like cars and modern medicine – both of which are great examples of how seemingly neutral technologies are not at all designed for women’s needs,” says Catherine Flick, reader in computing and social responsibility at De Montfort University.

Whether it’s using  – which are coded in with a traditionally male-focused bias – or to , where the majority of the training data is likely to have been men because they are overrepresented in such datasets, the risks of layering AI into the real world can be enormous.

It’s not just the risks of AI’s transformative effects that could more severely impact women. A recent forecast by McKinsey believes that women are  from their jobs by AI than men. It’s also how they’re used.

One of the biggest tech companies in the world, Amazon, experienced this in 2018, when they  an AI-powered recruitment system because the data used to train it was so male-biased that the AI penalised any mention of “women” in a job application. “Women were so underrepresented in the data that the model would essentially reject any time ‘women’ appeared,” says Sasha Luccioni, researcher in ethical and sustainable artificial intelligence at HuggingFace – a US company that develops tools for building applications using machine learning – and a board member of Women in Machine Learning. “The underlying issue behind that is the gender imbalance in tech.” 

There are  with at least one male author as there are papers with one female author, according to an analysis by the OECD. Just AI researchers worldwide are women. 

“There are very few women working in the field, very few women having a seat at the table,” says Luccioni. “And so obviously, this impacts the algorithms because algorithms carry values. They carry, implicitly, things that we care about.”

The issue is that, despite what many think, AI algorithms don’t conjure up their so-called “intelligence” from nowhere. They are shaped, implicitly and explicitly, by the design decisions made further up the pipeline of their creation. What data to train a model on, or what weights to place on different parts of the thinking – or, indeed, what problems to ask AI to tackle – can all be biased against the female experience from the start. “If you don’t have enough women at the table, you tend not to consider things that represent the values or lived experiences of women,” Luccioni says.

That lack of consideration includes thinking about the biases in the training data itself. It’s now known that many large language models (LLMs), including tools like ChatGPT, are trained on large swathes of the internet, including social network Reddit. Reddit is such a popular dataset for LLMs because it contains millions of users interacting with each other in natural language – enabling the conversational tone of tools like ChatGPT to be so human-like.

But there’s one problem. . So the conversations LLMs are mimicking are those you’d encounter among a majority group of men, rather than a finely-balanced group of people.

“For AI technologies to overcome this it’s not enough to just plaster over the diversity cracks,” says Flick. “The whole system, from data to engineers to c-suite, needs to be diversified and focused on ensuring harmful biases are not perpetuated by this technology.”

Flick has a simple message that gets to the core of the problem. “The usual focus on the engineering should be the lowest priority,” she says. “If the data going in isn’t good enough, it shouldn’t even get to the engineering stage.”

Sadly, it appears those warnings aren’t being heeded. Luccioni’s organisation, Hugging Face,  that analysed the output of text-to-image generators called Stable Bias. “The models will assume that whatever thing that you want to generate, it’ll be male,” says Luccioni. “An astronaut is going to be a white man, unless you say, ‘black female astronaut’.”

It’s not enough to assume that things will sort themselves out, warns Luccioni. “Given the fact that AI models are a reflection of our values, our society, and our choices, it’s really important to have more diversity in the people making these choices,” she says. It’s a mistake to believe that these are simple mathematical models that are free from any risk of human bias. “Actually, they reflect the choices we make as people training and using AI models.

“It’s really important to improve the gender gap,” she says, “and to contribute to improving AI models’ performance on non-males.”