Patient empowerment expert David Gilbert has sought help and advice from AI for his own mental health problems. In a new paper from Centre for Mental Health and InHealth Associates, Unequal benefits, unequal harms, he assesses the research on how the huge growth in the use of AI mental health tools can put people at risk and increase inequality.
The rapid expansion of AI mental health chatbots and self-guided AI systems has outpaced the development of robust mechanisms to protect people from harm, the paper says. Oversight is uncoordinated and there are significant gaps in evidence, accountability and patient safety. Without proper regulation, information content, quality and safety are at risk.
The paper also warns that while AI mental health tools may improve access and affordability for some people, benefits will not be distributed evenly and research shows that AI could exacerbate inequalities more widely, with the risks associated with generative AI mental health systems likely to fall disproportionately on those who are already vulnerable.
Large language models also absorb patterns that contain forms of structural discrimination. When used in contexts such as mental health support, these biases can be damaging, for example, by reinforcing stereotypes or generating responses that invalidate certain identities.
But there is not enough data about who uses what sort of AI tools withindifferent demographic and socioeconomic groups, with significant gaps in relation to deprivation, ethnicity, disability, neurodiversity, severe mental illness and experiences of trauma.
Centre for Mental Health and InHealth Associates are calling for more research into potential harm being generated amongst the most vulnerable.
Sam Dick, Associate Director of Policy at Centre for Mental Health, said: “This timely and valuable report reminds us that the mental health crisis is complex and broad, and cannot easily be ‘solved’ by new solutions such as increased use of digital and AI tools. They come with significant risks, alongside some opportunities, that must be considered and addressed so that no one is harmed by the rollout of untested and unsafe tools.”
David Gilbert, Director of InHealth Associates said: “I’m a mental health service user and one of millions of guinea pigs – intrigued by the benefits but concerned about the harms of AI mental health chatbots. I wanted to find out more and was shocked – for example by the lack of data about who uses what, the paucity of regulation and the complete disregard of patient-centred governance.
We are in thrall to this wave of tech. We have to balance that and do more to safeguard people, particularly those from marginalised groups who are at disproportionate risk of harm.”
