
AI Chatbots Could Support Mental Health but May Also Reinforce Harmful Thinking, Researcher Warns
Key Takeaways:
- A new framework from Curtin University proposes that AI chatbot conversations can either broaden a person’s thinking and support their mental health or narrow it into cycles of repeated thoughts and reassurance-seeking.
- The same chatbot could help one person while harming another, or affect the same person differently over time, depending on their emotional state, what they hope to gain and how the chatbot responds.
- Professor Warren Mansell argues developers should design chatbots that ask thoughtful questions and encourage reflection, rather than simply telling people what they want to hear; the framework is yet to be tested.
Two possible paths for AI conversations
Artificial intelligence (AI) chatbots could help people better understand and manage their mental health, but they could also reinforce harmful thoughts and increase emotional dependence, according to a researcher at Curtin University.
The paper, published in Digital Health, compares AI conversations that follow two possible paths. In the first, the conversation helps people explore their problems and consider new perspectives. In the second, it traps them in a cycle of repeated thoughts and reassurance.
The author, Professor of Mental Health Warren Mansell from the Curtin School of Population Health, has proposed a new framework to explain why AI conversations can have very different effects on people. According to the framework, these effects depend on a person’s emotional state, what they hope to gain from the interaction and how the chatbot responds.
Professor Mansell has a long track record in this area. He was the first to evaluate a therapy chatbot, back in 2013, and his research has focused on trying to measure and understand how a person’s attention either narrows or broadens during a conversation.
When chatbots can be helpful
Professor Mansell said chatbots could be helpful when they helped people to explore their feelings and question their assumptions, rather than simply agreeing with everything they said.
“AI conversations can move in very different directions depending on what a person brings to the interaction and how the chatbot responds,” Professor Mansell said.
“For some people, a chatbot may provide a space to express their feelings and explore problems without fear of judgement. For others, repeatedly receiving reassurance or having their existing beliefs confirmed could make it harder to see things differently.”
Mixed evidence on AI and mental health
Research into AI and mental health has so far produced contrasting results. Some studies suggest chatbots can reduce symptoms of depression and anxiety. Others have raised concerns about harmful responses, emotional dependence and AI reinforcing people’s existing beliefs, even when those beliefs could be damaging.
Professor Mansell said the same chatbot could potentially help one person while having a negative effect on another. It could even affect the same person differently depending on how they use it over time.
For example, a person experiencing anxiety might use a chatbot to explore their worries and consider alternative ways of looking at a situation. However, if the conversation repeatedly focuses on seeking reassurance or confirming their fears, it could narrow their thinking rather than help them work through the problem.
Greater risks for people who are already vulnerable
The paper suggests the risks could be particularly concerning for people who are already vulnerable and most in need of psychological support.
AI chatbots are often accessible, inexpensive and available at any time. However, Professor Mansell argues that their benefits may depend on whether they encourage meaningful reflection or simply provide immediate reassurance to keep people engaged.
What this means for chatbot design
Professor Mansell said developers should design chatbots that encourage people to think through their problems, ask thoughtful questions and consider different perspectives, rather than simply telling people what they want to hear.
The proposed framework also highlights the importance of understanding how conversations develop over time. Rather than judging a chatbot’s effectiveness solely on whether people feel better immediately after an interaction, it calls for closer attention to the direction a conversation takes.
For healthcare professionals, these findings are a reminder that the people they support may already be turning to AI tools between appointments. Understanding how these tools work, and where their limitations lie, is becoming an increasingly important part of practice. The College of Contemporary Health’s AI Essentials for Primary Care short course is designed to help clinicians build this understanding and approach AI in healthcare with confidence.
Advice for people using chatbots
Professor Mansell recommends that people approach AI conversations with a clear purpose. He also suggests they consider how their use of chatbots fits alongside other priorities and relationships in their lives.
Further research needed
The proposed framework is yet to be tested, and further research is needed to establish why some AI conversations may support mental health while others could increase psychological distress.
As AI becomes an increasingly accessible source of mental health support, Professor Mansell said understanding these differences would be important for developing safer and more effective technology.
The paper, ‘Up the mountain or down the rabbit hole? A simple rubric to conceptualise the benefits and risks of AI conversations as dynamic changes in awareness’, was published in Digital Health.
CCH insight
As more people turn to AI chatbots for support, healthcare professionals need to understand both the potential and the pitfalls of these tools. Our AI Essentials for Primary Care CPD short course helps clinicians develop the knowledge to navigate AI in practice safely and confidently.
Explore AI Essentials for Primary Care →
Source: Digital Health
Read More
People Like AI Mental Health Chatbots. Whether They Help Is Another Question
Key Takeaways:
- Across 21 studies in 11 countries, people using generative AI mental health chatbots reported high satisfaction and found them convenient and accessible.
- Personalisation and empathy did not reliably translate into better clinical outcomes, and engagement often faded over time.
- The evidence base remains early-stage, leaving safety, equity and crisis response unresolved.
A treatment gap that digital tools are being asked to fill
Generative artificial intelligence (GenAI) chatbots designed to support mental health are winning people over on experience, but the research needed to establish whether they are safe and clinically effective has not kept pace. That is the central finding of a review, currently in press in the journal npj Digital Medicine, which examined the user experience (UX) and intervention design of GenAI mental health chatbots.
The context for this work is a substantial and persistent shortfall in care. Around 25% of people worldwide experience a mental health problem, yet approximately 85% do not receive adequate treatment. The reasons are varied and overlapping: stigma, cost, shortages of trained professionals, geographic distance from services and structural inequities, among others. As the prevalence of mental health conditions has grown while treatment gaps have remained, attention has turned towards innovative models of delivery – and digital tools, with their scalability and convenience, have become a focus of that search.
Digital mental health interventions deliver treatment or support through a range of channels, including chatbots, websites, mobile applications and wearables. Conversational agents, more commonly known as chatbots, are applications that simulate human dialogue using machine learning and natural language processing algorithms.
From scripted responses to open-ended conversation
Traditional mental health chatbots deliver pre-scripted therapeutic content through rules-based or retrieval-based systems. Their strength is predictability, but that same design limits their capacity to personalise support or to recognise what an individual actually needs in the moment.
Chatbots built on large language models (LLMs) work differently. They can simulate core aspects of a therapeutic encounter, including personalised suggestions and empathetic reflections. That flexibility comes with a trade-off. GenAI systems may produce responses that are incorrect or inappropriate, and their open-ended conversational capacity makes intervention design both more consequential and more complex than it is for rules-based systems. When a system can say almost anything, design decisions carry considerably more weight.
How the review was carried out
The researchers set out to map the design characteristics and UX outcomes of interventions involving GenAI mental health chatbots. They began with a systematic literature search to identify studies covering the design and deployment of such tools. Reviews, editorials, media articles and commentaries were excluded.
In total, 21 studies were selected, conducted across 11 countries between 2023 and 2025. The largest numbers came from China and the United Kingdom, followed by the United States. The included work spanned a wide range of maturity, from early-stage prototype evaluations through to clinical trials, with one real-world implementation study.
Most studies recruited general or clinical adult populations, including older people living with dementia. Others involved simulated users or university students. Sample sizes ranged from as few as five participants to as many as 527. Across the body of evidence, there was substantial heterogeneity in outcome measures, and most interventions remained at an early stage of development – two features that shape how much can reasonably be concluded from the literature as it stands.
What the interventions were designed to do
Chatbot interventions most often targeted depression and anxiety, and tended to adopt shared therapeutic mechanisms, including mindfulness, emotion regulation and cognitive restructuring. Some systems were oriented towards mental well-being, stress and loneliness, emphasising general support and preventive care rather than treatment for a specific condition. Others addressed eating disorders, post-traumatic stress disorder and dementia.
The dementia-focused interventions are worth distinguishing. Rather than attempting to address the central neurological features of the condition, they targeted its related psychological dimensions – carer burnout, psychological distress and loneliness among them.
Most interventions were grounded in cognitive-behavioural therapy principles. The specific techniques drawn upon included behavioural activation, psychoeducation, Socratic dialogue, acceptance and commitment therapy, cognitive restructuring and mindfulness.
How the tools were delivered
Interventions varied in frequency, delivery modality and duration. The majority were short-term, running from two to eight weeks. Most were deployed through web-based interfaces and mobile applications, while some were delivered via messaging or social media platforms – meeting people on services they already used rather than asking them to adopt something new.
About 67% of interventions were non-embodied, text-based chatbots. The remainder used voice, avatar-based, augmented reality or other multimodal forms of interaction, with the intention of improving engagement and realism.
What people made of them
All but two of the studies evaluated at least one UX domain. The majority relied on quantitative measures, typically Likert scales, while some gathered qualitative feedback through open-ended questions and semi-structured interviews.
User satisfaction and acceptability were the most commonly reported outcomes. Across studies, participants described the interventions as convenient and accessible, with acceptability generally rated moderate-to-high and reported satisfaction high.
Half of the studies examined usability, using qualitative feedback, the System Usability Scale or Likert scales. Interface design, interaction mode and deployment platform were all observed, alongside differences in usability between studies. A clear preference emerged for free-flowing chat interfaces and customisable features over predefined options. At the same time, some interventions had an unclear scope or limited functionality, leaving people uncertain about what the chatbot could actually do for them.
Usability, engagement and the drop-off problem
Only some studies reported objective utilisation and engagement metrics, such as session frequency, interaction duration, retention over time and task completion. Where these were captured, attrition patterns frequently emerged over time in repeated-measures designs. Uptake in multi-week interventions was often high at the outset before declining – a pattern familiar across digital health more broadly, and one that matters a great deal for interventions whose therapeutic logic depends on sustained practice.
Personalisation and perceived benefit
Most chatbots featured some form of personalisation, reflecting their capacity to adapt conversations and interfaces in response to previous interactions and a person’s emotional state. The most common approach was emotion detection paired with adaptive interaction, allowing people to receive tailored responses and empathetic reflections.
Perceived impact was not consistently measured as a standalone metric. It was more usually folded into qualitative feedback or broader UX evaluations. In the intervention that produced the most granular data, the most frequently reported benefit was improved clarity and awareness.
Where empathy stops being enough
The review’s more cautionary finding is that personalisation and empathy did not consistently translate into stronger clinical outcomes or sustained use. Feeling supported and being helped are not the same thing, and the studies reviewed do not yet demonstrate a reliable link between the two.
Some people reported responses that were repetitive, generic or contextually misaligned. Others raised concerns about over-reliance on chatbots, reduced human contact, data privacy and whether these systems can respond appropriately when someone is in crisis. Inaccurate or clinically misaligned outputs were also linked to an erosion of trust and to disengagement in several studies.
For healthcare professionals, the practical question is less whether these tools have promise than how to appraise them – knowing what a given system is grounded in, where its limits sit and when a conversation needs to move to a human. That judgement is increasingly treated as a core clinical competency, and it sits at the centre of CPD training on the everyday, ethical use of AI in practice.
Design features linked to a better experience
The authors identified several design features associated with better UX outcomes, while being careful to note that these were associations rather than demonstrated causes. They included:
- Deployment on platforms people already knew and used
- Richer interaction modalities beyond plain text
- Integration into existing care pathways
- Personalisation
- Grounding in domain knowledge
- Structured delivery
- Proactive outreach
- Co-design with both experts and end users
The predominance of early-stage studies, combined with limited direct comparative analyses, prevented firm conclusions about which of these features genuinely improved user experience.
What needs to happen next
Taken together, the review suggests that GenAI chatbots have meaningful potential to deliver tailored, empathetic mental health support, and that their acceptability among the people who use them is promising. That is a real finding, and not a small one given the scale of unmet need.
Significant challenges remain, however. Standardising how UX is assessed, grounding intervention design in the needs and preferences of the people who will use these tools, and sustaining engagement beyond the first few weeks all stand out as unresolved. Addressing them, the authors argue, will require co-design with experts and users, validated UX metrics applied in long-term studies, transparent reporting standards, independent evaluation, clearer reporting of model design and training data, and stronger attention to safety, equity and the limits of crisis response.
CCH insight
Generative AI tools are arriving in patient-facing care faster than the evidence base supporting them, which puts the burden of appraisal on clinicians. CCH’s CPD-accredited short course AI Essentials for Primary Care: Tools, Ethics and Everyday Applications covers the practical and ethical judgement this requires – what these tools can and cannot do, where the risks sit, and how to use them safely in day-to-day practice.
Find out more about AI Essentials for Primary Care →

AI-Supported Digital Care Improves Rheumatoid Arthritis Outcomes After Hospital Discharge
Key Takeaways:
- A nurse-led, AI-assisted digital platform reduced disease activity and improved physical function more than routine care over six months.
- People using the platform showed higher medication adherence and markedly greater satisfaction with their care.
- Real-time monitoring enabled earlier detection of problems and more personalised support between clinic visits.
The challenge of care after discharge
Rheumatoid arthritis is a long-term autoimmune condition that causes joint pain, swelling and a progressive loss of function. Managing it well once people leave hospital is often difficult, because symptoms can fluctuate and regular follow-up is not always easy to arrange. A recent real-world study set out to test whether an artificial intelligence (AI)-assisted digital care platform could improve outcomes for people living with the condition after discharge.
How the study was designed
The study, published in JMIR Medical Informatics and conducted by Ziyun Zhang, PhD, and colleagues at Tongji Hospital, followed 341 people with rheumatoid arthritis over a six-month period in a real clinical setting.
Participants were divided into two groups. One group received standard post-discharge care, while the other used a nurse-led digital management platform supported by AI. The platform allowed people to report symptoms, fatigue, medication use, laboratory results and emotional wellbeing through a smartphone app. This information was stored securely and analysed in real time. When the system detected concerning changes, healthcare staff were alerted so that they could respond quickly. Nurses and health coaches also provided ongoing education and personalised support.
The researchers focused on how disease activity, physical function, medication adherence and satisfaction changed over time, using standard clinical tools to measure disease severity and disability.
What the platform achieved
After six months, both groups showed some improvement, but the differences between them were notable. People using the AI-supported platform experienced a greater reduction in disease activity scores, meaning their arthritis was better controlled. They also showed significant improvements in physical function compared with those receiving routine care alone.
Medication adherence was higher in the digital care group, with more people taking their medicines as prescribed. Satisfaction levels were significantly higher too, with a large majority of those using the platform reporting that they were very satisfied with their care experience, compared with the standard care group.
What it means for long-term management
The authors conclude that the combination of AI monitoring, nurse-led support and continuous digital engagement helped to improve both clinical outcomes and the experience of care. The system made it easier to detect problems early, encourage medication use and provide more personalised care between clinic visits.
Overall, the study suggests that digital health platforms could play an important role in improving the long-term management of rheumatoid arthritis, particularly by keeping people more closely connected to their care teams after they leave hospital.
Read More
Digital Health Tools Show Early Promise for Infant Feeding and Sleep, UMass Chan Research Finds
Key Takeaways:
- Families who completed three or more visits with the virtual feeding service SimpliFed provided breast milk for nearly 16 weeks longer than those who did not use it.
- Infants whose parents engaged most actively with the AI-powered sleep app Huckleberry slept around 90 minutes longer during their longest overnight stretch.
- Lower uptake among Spanish-speaking and publicly insured families underlines the need to make digital health support equitable rather than exclusionary.
Studying whether technology can support new parents
Researchers at UMass Chan Medical School are investigating whether digital tools for infant feeding, sleep and other early parenting challenges can improve health outcomes and widen access to support for families.
Among them is Nisha Fahey, DO, MSc’21, assistant professor of pediatrics and principal investigator on research examining how digital health interventions can support families during the critical first year of a child’s life. Dr Fahey has led two pilot studies evaluating virtual lactation support and an artificial intelligence–powered infant sleep application, carried out in collaboration with the Department of Medicine’s Program in Digital Medicine. That programme is led by Apurv Soni, MD, PhD’21, assistant professor of medicine and the programme’s co-director, who serves as multiprincipal investigator on the work.
As a paediatrician, Dr Fahey hears the same questions from new parents every day: Is my baby feeding enough? Are they sleeping enough? And where can I turn for help when I need it?
“These technologies already exist. Families are accessing them and using them,” said Fahey. “As researchers and healthcare providers, it’s our responsibility to understand their impact and think about how they can be integrated into healthcare in a way that is equitable and reaches all families.”
Virtual feeding support and longer breastfeeding
The first study examined SimpliFed, a virtual infant-feeding support platform that gives families on-demand access to certified lactation consultants and feeding specialists. Researchers enrolled 200 pregnant and postpartum individuals through UMass Memorial Health’s obstetrics clinics and followed them through the first year of their infant’s life.
The study assessed infant growth and development, maternal mental health, healthcare utilisation and feeding practices. Researchers found that participants who completed three or more visits with SimpliFed provided breast milk for nearly 16 weeks longer than participants who did not use the service.
The findings also drew attention to important equity considerations. Uptake was lower among Spanish-speaking families and among publicly insured participants, underscoring the need to ensure that digital health interventions reach populations that have historically faced barriers to care.
“If health systems are going to deploy these tools broadly, we need to pay special attention to making sure all patients and families can access them,” Fahey said. “The goal is to close gaps in care, not widen them.”
An AI sleep app and longer overnight rest
A second pilot study evaluated Huckleberry, a mobile app that allows parents to track infant sleep and uses artificial intelligence to predict optimal nap and bedtime schedules. This study was funded by an NIH grant focused on point-of-care technologies for heart, lung, blood and sleep disorders.
The study enrolled approximately 80 families with infants under 12 months who are beneficiaries of UMass Memorial’s MassHealth Accountable Care Organization. Participants used the app for three months while researchers tracked engagement and measured infant sleep, parental sleep and parental mental health.
Among families who engaged most actively with the app, infants experienced longer consolidated overnight sleep. Researchers found that infants in the high-engagement group slept approximately 90 minutes longer during their longest stretch of overnight sleep, compared with participants who used the app less frequently.
The researchers also found that families in a population often underrepresented in digital health research were willing and able to engage with the technology. About half of participants were classified as highly engaged users, and most reported that they found the app useful and would recommend it to other families.
Recognising the limitations
The studies also revealed some limitations. While many families reported positive experiences, others described challenges with tracking data consistently or navigating app features while caring for a young infant.
For Dr Fahey, those findings reinforce the importance of viewing digital health as a complement to, rather than a replacement of, traditional care.
“Digital technologies offer an on-demand pathway for information and support,” she said. “The goal is to make both digital and in-person care as accessible as possible and empower families to choose what works best for them.”
Building evidence for the future of care
The research was made possible through collaborations across UMass Chan, including faculty in the Program in Digital Medicine, the Department of Obstetrics & Gynecology, the Department of Psychiatry & Behavioral Health, and the Department of Pediatrics.
“Parents are seeking out digital health apps on their own,” Fahey said. “Building evidence around their benefits and understanding their limitations helps us determine whether they can become trusted parts of care in the future.”
Source: UMass Chan Medical School
Read More