AI Chatbots Could Support Mental Health but May Also Reinforce Harmful Thinking, Researcher Warns
Key Takeaways:
- A new framework from Curtin University proposes that AI chatbot conversations can either broaden a person’s thinking and support their mental health or narrow it into cycles of repeated thoughts and reassurance-seeking.
- The same chatbot could help one person while harming another, or affect the same person differently over time, depending on their emotional state, what they hope to gain and how the chatbot responds.
- Professor Warren Mansell argues developers should design chatbots that ask thoughtful questions and encourage reflection, rather than simply telling people what they want to hear; the framework is yet to be tested.
Two possible paths for AI conversations
Artificial intelligence (AI) chatbots could help people better understand and manage their mental health, but they could also reinforce harmful thoughts and increase emotional dependence, according to a researcher at Curtin University.
The paper, published in Digital Health, compares AI conversations that follow two possible paths. In the first, the conversation helps people explore their problems and consider new perspectives. In the second, it traps them in a cycle of repeated thoughts and reassurance.
The author, Professor of Mental Health Warren Mansell from the Curtin School of Population Health, has proposed a new framework to explain why AI conversations can have very different effects on people. According to the framework, these effects depend on a person’s emotional state, what they hope to gain from the interaction and how the chatbot responds.
Professor Mansell has a long track record in this area. He was the first to evaluate a therapy chatbot, back in 2013, and his research has focused on trying to measure and understand how a person’s attention either narrows or broadens during a conversation.
When chatbots can be helpful
Professor Mansell said chatbots could be helpful when they helped people to explore their feelings and question their assumptions, rather than simply agreeing with everything they said.
“AI conversations can move in very different directions depending on what a person brings to the interaction and how the chatbot responds,” Professor Mansell said.
“For some people, a chatbot may provide a space to express their feelings and explore problems without fear of judgement. For others, repeatedly receiving reassurance or having their existing beliefs confirmed could make it harder to see things differently.”
Mixed evidence on AI and mental health
Research into AI and mental health has so far produced contrasting results. Some studies suggest chatbots can reduce symptoms of depression and anxiety. Others have raised concerns about harmful responses, emotional dependence and AI reinforcing people’s existing beliefs, even when those beliefs could be damaging.
Professor Mansell said the same chatbot could potentially help one person while having a negative effect on another. It could even affect the same person differently depending on how they use it over time.
For example, a person experiencing anxiety might use a chatbot to explore their worries and consider alternative ways of looking at a situation. However, if the conversation repeatedly focuses on seeking reassurance or confirming their fears, it could narrow their thinking rather than help them work through the problem.
Greater risks for people who are already vulnerable
The paper suggests the risks could be particularly concerning for people who are already vulnerable and most in need of psychological support.
AI chatbots are often accessible, inexpensive and available at any time. However, Professor Mansell argues that their benefits may depend on whether they encourage meaningful reflection or simply provide immediate reassurance to keep people engaged.
What this means for chatbot design
Professor Mansell said developers should design chatbots that encourage people to think through their problems, ask thoughtful questions and consider different perspectives, rather than simply telling people what they want to hear.
The proposed framework also highlights the importance of understanding how conversations develop over time. Rather than judging a chatbot’s effectiveness solely on whether people feel better immediately after an interaction, it calls for closer attention to the direction a conversation takes.
For healthcare professionals, these findings are a reminder that the people they support may already be turning to AI tools between appointments. Understanding how these tools work, and where their limitations lie, is becoming an increasingly important part of practice. The College of Contemporary Health’s AI Essentials for Primary Care short course is designed to help clinicians build this understanding and approach AI in healthcare with confidence.
Advice for people using chatbots
Professor Mansell recommends that people approach AI conversations with a clear purpose. He also suggests they consider how their use of chatbots fits alongside other priorities and relationships in their lives.
Further research needed
The proposed framework is yet to be tested, and further research is needed to establish why some AI conversations may support mental health while others could increase psychological distress.
As AI becomes an increasingly accessible source of mental health support, Professor Mansell said understanding these differences would be important for developing safer and more effective technology.
The paper, ‘Up the mountain or down the rabbit hole? A simple rubric to conceptualise the benefits and risks of AI conversations as dynamic changes in awareness’, was published in Digital Health.
CCH insight
As more people turn to AI chatbots for support, healthcare professionals need to understand both the potential and the pitfalls of these tools. Our AI Essentials for Primary Care CPD short course helps clinicians develop the knowledge to navigate AI in practice safely and confidently.
Explore AI Essentials for Primary Care →
Source: Digital Health
Read More
People Like AI Mental Health Chatbots. Whether They Help Is Another Question
Key Takeaways:
- Across 21 studies in 11 countries, people using generative AI mental health chatbots reported high satisfaction and found them convenient and accessible.
- Personalisation and empathy did not reliably translate into better clinical outcomes, and engagement often faded over time.
- The evidence base remains early-stage, leaving safety, equity and crisis response unresolved.
A treatment gap that digital tools are being asked to fill
Generative artificial intelligence (GenAI) chatbots designed to support mental health are winning people over on experience, but the research needed to establish whether they are safe and clinically effective has not kept pace. That is the central finding of a review, currently in press in the journal npj Digital Medicine, which examined the user experience (UX) and intervention design of GenAI mental health chatbots.
The context for this work is a substantial and persistent shortfall in care. Around 25% of people worldwide experience a mental health problem, yet approximately 85% do not receive adequate treatment. The reasons are varied and overlapping: stigma, cost, shortages of trained professionals, geographic distance from services and structural inequities, among others. As the prevalence of mental health conditions has grown while treatment gaps have remained, attention has turned towards innovative models of delivery – and digital tools, with their scalability and convenience, have become a focus of that search.
Digital mental health interventions deliver treatment or support through a range of channels, including chatbots, websites, mobile applications and wearables. Conversational agents, more commonly known as chatbots, are applications that simulate human dialogue using machine learning and natural language processing algorithms.
From scripted responses to open-ended conversation
Traditional mental health chatbots deliver pre-scripted therapeutic content through rules-based or retrieval-based systems. Their strength is predictability, but that same design limits their capacity to personalise support or to recognise what an individual actually needs in the moment.
Chatbots built on large language models (LLMs) work differently. They can simulate core aspects of a therapeutic encounter, including personalised suggestions and empathetic reflections. That flexibility comes with a trade-off. GenAI systems may produce responses that are incorrect or inappropriate, and their open-ended conversational capacity makes intervention design both more consequential and more complex than it is for rules-based systems. When a system can say almost anything, design decisions carry considerably more weight.
How the review was carried out
The researchers set out to map the design characteristics and UX outcomes of interventions involving GenAI mental health chatbots. They began with a systematic literature search to identify studies covering the design and deployment of such tools. Reviews, editorials, media articles and commentaries were excluded.
In total, 21 studies were selected, conducted across 11 countries between 2023 and 2025. The largest numbers came from China and the United Kingdom, followed by the United States. The included work spanned a wide range of maturity, from early-stage prototype evaluations through to clinical trials, with one real-world implementation study.
Most studies recruited general or clinical adult populations, including older people living with dementia. Others involved simulated users or university students. Sample sizes ranged from as few as five participants to as many as 527. Across the body of evidence, there was substantial heterogeneity in outcome measures, and most interventions remained at an early stage of development – two features that shape how much can reasonably be concluded from the literature as it stands.
What the interventions were designed to do
Chatbot interventions most often targeted depression and anxiety, and tended to adopt shared therapeutic mechanisms, including mindfulness, emotion regulation and cognitive restructuring. Some systems were oriented towards mental well-being, stress and loneliness, emphasising general support and preventive care rather than treatment for a specific condition. Others addressed eating disorders, post-traumatic stress disorder and dementia.
The dementia-focused interventions are worth distinguishing. Rather than attempting to address the central neurological features of the condition, they targeted its related psychological dimensions – carer burnout, psychological distress and loneliness among them.
Most interventions were grounded in cognitive-behavioural therapy principles. The specific techniques drawn upon included behavioural activation, psychoeducation, Socratic dialogue, acceptance and commitment therapy, cognitive restructuring and mindfulness.
How the tools were delivered
Interventions varied in frequency, delivery modality and duration. The majority were short-term, running from two to eight weeks. Most were deployed through web-based interfaces and mobile applications, while some were delivered via messaging or social media platforms – meeting people on services they already used rather than asking them to adopt something new.
About 67% of interventions were non-embodied, text-based chatbots. The remainder used voice, avatar-based, augmented reality or other multimodal forms of interaction, with the intention of improving engagement and realism.
What people made of them
All but two of the studies evaluated at least one UX domain. The majority relied on quantitative measures, typically Likert scales, while some gathered qualitative feedback through open-ended questions and semi-structured interviews.
User satisfaction and acceptability were the most commonly reported outcomes. Across studies, participants described the interventions as convenient and accessible, with acceptability generally rated moderate-to-high and reported satisfaction high.
Half of the studies examined usability, using qualitative feedback, the System Usability Scale or Likert scales. Interface design, interaction mode and deployment platform were all observed, alongside differences in usability between studies. A clear preference emerged for free-flowing chat interfaces and customisable features over predefined options. At the same time, some interventions had an unclear scope or limited functionality, leaving people uncertain about what the chatbot could actually do for them.
Usability, engagement and the drop-off problem
Only some studies reported objective utilisation and engagement metrics, such as session frequency, interaction duration, retention over time and task completion. Where these were captured, attrition patterns frequently emerged over time in repeated-measures designs. Uptake in multi-week interventions was often high at the outset before declining – a pattern familiar across digital health more broadly, and one that matters a great deal for interventions whose therapeutic logic depends on sustained practice.
Personalisation and perceived benefit
Most chatbots featured some form of personalisation, reflecting their capacity to adapt conversations and interfaces in response to previous interactions and a person’s emotional state. The most common approach was emotion detection paired with adaptive interaction, allowing people to receive tailored responses and empathetic reflections.
Perceived impact was not consistently measured as a standalone metric. It was more usually folded into qualitative feedback or broader UX evaluations. In the intervention that produced the most granular data, the most frequently reported benefit was improved clarity and awareness.
Where empathy stops being enough
The review’s more cautionary finding is that personalisation and empathy did not consistently translate into stronger clinical outcomes or sustained use. Feeling supported and being helped are not the same thing, and the studies reviewed do not yet demonstrate a reliable link between the two.
Some people reported responses that were repetitive, generic or contextually misaligned. Others raised concerns about over-reliance on chatbots, reduced human contact, data privacy and whether these systems can respond appropriately when someone is in crisis. Inaccurate or clinically misaligned outputs were also linked to an erosion of trust and to disengagement in several studies.
For healthcare professionals, the practical question is less whether these tools have promise than how to appraise them – knowing what a given system is grounded in, where its limits sit and when a conversation needs to move to a human. That judgement is increasingly treated as a core clinical competency, and it sits at the centre of CPD training on the everyday, ethical use of AI in practice.
Design features linked to a better experience
The authors identified several design features associated with better UX outcomes, while being careful to note that these were associations rather than demonstrated causes. They included:
- Deployment on platforms people already knew and used
- Richer interaction modalities beyond plain text
- Integration into existing care pathways
- Personalisation
- Grounding in domain knowledge
- Structured delivery
- Proactive outreach
- Co-design with both experts and end users
The predominance of early-stage studies, combined with limited direct comparative analyses, prevented firm conclusions about which of these features genuinely improved user experience.
What needs to happen next
Taken together, the review suggests that GenAI chatbots have meaningful potential to deliver tailored, empathetic mental health support, and that their acceptability among the people who use them is promising. That is a real finding, and not a small one given the scale of unmet need.
Significant challenges remain, however. Standardising how UX is assessed, grounding intervention design in the needs and preferences of the people who will use these tools, and sustaining engagement beyond the first few weeks all stand out as unresolved. Addressing them, the authors argue, will require co-design with experts and users, validated UX metrics applied in long-term studies, transparent reporting standards, independent evaluation, clearer reporting of model design and training data, and stronger attention to safety, equity and the limits of crisis response.
CCH insight
Generative AI tools are arriving in patient-facing care faster than the evidence base supporting them, which puts the burden of appraisal on clinicians. CCH’s CPD-accredited short course AI Essentials for Primary Care: Tools, Ethics and Everyday Applications covers the practical and ethical judgement this requires – what these tools can and cannot do, where the risks sit, and how to use them safely in day-to-day practice.
Find out more about AI Essentials for Primary Care →