AI Chatbots Could Support Mental Health but May Also Reinforce Harmful Thinking, Researcher Warns
Key Takeaways:
- A new framework from Curtin University proposes that AI chatbot conversations can either broaden a person’s thinking and support their mental health or narrow it into cycles of repeated thoughts and reassurance-seeking.
- The same chatbot could help one person while harming another, or affect the same person differently over time, depending on their emotional state, what they hope to gain and how the chatbot responds.
- Professor Warren Mansell argues developers should design chatbots that ask thoughtful questions and encourage reflection, rather than simply telling people what they want to hear; the framework is yet to be tested.
Two possible paths for AI conversations
Artificial intelligence (AI) chatbots could help people better understand and manage their mental health, but they could also reinforce harmful thoughts and increase emotional dependence, according to a researcher at Curtin University.
The paper, published in Digital Health, compares AI conversations that follow two possible paths. In the first, the conversation helps people explore their problems and consider new perspectives. In the second, it traps them in a cycle of repeated thoughts and reassurance.
The author, Professor of Mental Health Warren Mansell from the Curtin School of Population Health, has proposed a new framework to explain why AI conversations can have very different effects on people. According to the framework, these effects depend on a person’s emotional state, what they hope to gain from the interaction and how the chatbot responds.
Professor Mansell has a long track record in this area. He was the first to evaluate a therapy chatbot, back in 2013, and his research has focused on trying to measure and understand how a person’s attention either narrows or broadens during a conversation.
When chatbots can be helpful
Professor Mansell said chatbots could be helpful when they helped people to explore their feelings and question their assumptions, rather than simply agreeing with everything they said.
“AI conversations can move in very different directions depending on what a person brings to the interaction and how the chatbot responds,” Professor Mansell said.
“For some people, a chatbot may provide a space to express their feelings and explore problems without fear of judgement. For others, repeatedly receiving reassurance or having their existing beliefs confirmed could make it harder to see things differently.”
Mixed evidence on AI and mental health
Research into AI and mental health has so far produced contrasting results. Some studies suggest chatbots can reduce symptoms of depression and anxiety. Others have raised concerns about harmful responses, emotional dependence and AI reinforcing people’s existing beliefs, even when those beliefs could be damaging.
Professor Mansell said the same chatbot could potentially help one person while having a negative effect on another. It could even affect the same person differently depending on how they use it over time.
For example, a person experiencing anxiety might use a chatbot to explore their worries and consider alternative ways of looking at a situation. However, if the conversation repeatedly focuses on seeking reassurance or confirming their fears, it could narrow their thinking rather than help them work through the problem.
Greater risks for people who are already vulnerable
The paper suggests the risks could be particularly concerning for people who are already vulnerable and most in need of psychological support.
AI chatbots are often accessible, inexpensive and available at any time. However, Professor Mansell argues that their benefits may depend on whether they encourage meaningful reflection or simply provide immediate reassurance to keep people engaged.
What this means for chatbot design
Professor Mansell said developers should design chatbots that encourage people to think through their problems, ask thoughtful questions and consider different perspectives, rather than simply telling people what they want to hear.
The proposed framework also highlights the importance of understanding how conversations develop over time. Rather than judging a chatbot’s effectiveness solely on whether people feel better immediately after an interaction, it calls for closer attention to the direction a conversation takes.
For healthcare professionals, these findings are a reminder that the people they support may already be turning to AI tools between appointments. Understanding how these tools work, and where their limitations lie, is becoming an increasingly important part of practice. The College of Contemporary Health’s AI Essentials for Primary Care short course is designed to help clinicians build this understanding and approach AI in healthcare with confidence.
Advice for people using chatbots
Professor Mansell recommends that people approach AI conversations with a clear purpose. He also suggests they consider how their use of chatbots fits alongside other priorities and relationships in their lives.
Further research needed
The proposed framework is yet to be tested, and further research is needed to establish why some AI conversations may support mental health while others could increase psychological distress.
As AI becomes an increasingly accessible source of mental health support, Professor Mansell said understanding these differences would be important for developing safer and more effective technology.
The paper, ‘Up the mountain or down the rabbit hole? A simple rubric to conceptualise the benefits and risks of AI conversations as dynamic changes in awareness’, was published in Digital Health.
CCH insight
As more people turn to AI chatbots for support, healthcare professionals need to understand both the potential and the pitfalls of these tools. Our AI Essentials for Primary Care CPD short course helps clinicians develop the knowledge to navigate AI in practice safely and confidently.
Explore AI Essentials for Primary Care →
Source: Digital Health
Read More