Even AI Thinks It’s a Man’s Job
The first hurdle may arise even before an application is submitted. A woman asks artificial intelligence to help her write her CV: the system chooses words, highlights qualities and builds a profile. Then another system, perhaps based on the same model, evaluates that text. If both reproduce gender stereotypes, the risk is a double filter: bias enters the application and is reinforced by the person selecting it.
This is the mechanism at the heart of Automating Inequality? Gender Bias in AI-mediated Labour Markets, published in PNAS, the Proceedings of the National Academy of Sciences. The study is authored by Kenza Elass (ENS de Lyon); Germain Gauthier, Paola Profeta and Debora Nozza (Bocconi).
The research challenges an implicit promise of automation: that entrusting more stages of the job search to an algorithm necessarily makes the process more impartial.
Stereotypes creep into CVs
The researchers conducted controlled experiments on six language models — GPT 5.4, Claude Opus 4.6, DeepSeek 3.2, Mistral Large 3, Grok 4.1 and Llama 3.3 70B — and 98 occupations, using the US labour market as a reference. In the initial phase alone, they generated 35,280 CVs, randomly assigning the candidate’s gender. They then asked the systems to suggest a salary and assign an employability score, isolating the effect of the CV’s content from that of the gender with which it was presented.
The CVs produced are not equivalent. Those written for women contain more references to being helpful towards others, to contributing to the life of the organization, and to tasks that are unlikely to lead to promotion. Even when explicit references to gender are removed, a classifier can still recognize whether a text was generated for a woman or a man in 81 per cent of cases. The stereotype, therefore, remains in the words.
“We are used to judging these tools by the quality of the writing: a text that reads smoothly and professionally also strikes us as a good result. But linguistic quality and fairness are different things,” observes Debora Nozza, Assistant Professor in the Department of Computing Sciences at Bocconi University. “Anyone developing a CV-writing assistant must also ask themselves what opportunities it gives people to present themselves and which, on the other hand, it restricts through seemingly innocuous choices.”
The problem therefore arises even in a tool designed to help jobseekers. When the initial information is scarce, the model has more scope to fill in the gaps with stereotypical representations. Providing more detailed data on education and experience reduces this margin and mitigates the impact of the generated content on subsequent assessments. It does not, however, eliminate disparities in assessment.
The apparent parity of the averages
In selection scores, the average gap between men and women is close to zero and not statistically significant. But this apparent parity masks contrasting results: in occupations where women make up less than 10 per cent of the workforce, female candidates receive scores that are approximately 2.7 per cent lower; in those where women account for over 80 per cent, they gain an advantage of a similar magnitude.
The risk is that this will reinforce the division between professions considered ‘male’ and ‘female’. Even small differences in scores can affect access to opportunities: in a selection simulation devised by the authors, in occupations with fewer than 10 per cent women, the probability of a female candidate passing the initial screening is 4.8 percentage points lower than that of a man.
“A system can accurately describe the labor market and, precisely for this reason, become problematic when used to guide decisions,” explains Germain Gauthier, Assistant Professor in the Department of Social and Political Sciences at Bocconi. “The historical prevalence of more men or more women in a profession is not a measure of the suitability of the next candidate. The transition from description to recommendation is the point that needs to be scrutinized.”
Salary recommendations also reveal disparities. When gender is explicitly stated, recommendations for women are on average around 1.6 per cent lower, with greater penalties in professions where the observed pay gap is wider. However, the result depends on the model used and how gender is communicated. When the authors replace the explicit statement ‘I am a man’ or ‘I am a woman’ with a male or female name, in five of the six models the penalties for female candidates still tend to increase in professions with higher pay gaps. GPT 5.4, on the other hand, shows the opposite trend, which offsets that of the others and causes this relationship to disappear in the aggregate result of the six models. The phenomenon is therefore widespread, but not universal.
“The salary suggested by a digital assistant can become the starting point for a negotiation,” observes Kenza Elass, a fellow at Bocconi’s Research Lab on Gender Equality. “If that reference incorporates a gender bias, the female candidate risks taking it with her to the negotiating table, without realizing that the advice would have been different for a man.”
Why an average is not enough
The implications for businesses and public decision-makers stem from this: assessing the fairness of a system solely on the basis of the average outcome can leave the most significant disparities unseen. Separate checks are needed for each profession, and assessments must be carried out throughout the entire process – from the drafting of materials to evaluation – and repeated whenever the models are updated.
Simply adding a reassuring instruction is not enough. In the experiments, asking the model to be strictly gender-neutral leaves the problems largely intact. Inviting it, on the other hand, to favor female candidates produces a strong, generalized correction, rather than an intervention targeted at the imbalances. The ease with which an instruction can be modified does not, therefore, translate into the ability to control its effects.
“The policy question is who should demonstrate that these tools operate fairly, through what checks and at what stage,” emphasizes Paola Profeta, director of the Research Lab on Gender Equality at Bocconi University. “Checks should form part of the conditions under which a company selects and implements a technology, alongside costs and performance. Intervening only when an individual manages to recognize and challenge a disparity places too great a burden on their shoulders.”
The study measures the behavior of the models under experimental conditions, not their effects on actual recruitment and salaries. Furthermore, the experiments are conducted in English and relate to the US context: field tests and studies in other languages will be needed to gauge the scope of the mechanisms identified.
The message for those introducing these tools is clear, however. Impartiality must be demonstrated even in the less visible stages – those in which decisions are made about how to present a person even before they are assessed. Otherwise, automation risks providing old inequalities with a new infrastructure.