
Pharmaceutical Ads Can Sway AI Treatment Recommendations – Often Without Anyone Noticing
Key Takeaways:
- When a clearly labelled drug advertisement appeared before a clinical case, 12 AI models selected the advertised drug in 47.6 per cent of responses, compared with 35 per cent without an ad.
- The answers often still appeared medically sound, and models explicitly mentioned the ad in only 29.9 per cent of a subset of free-text responses – so the influence could be easy to miss.
- The researchers recommend testing AI systems with and without promotional content, disclosing when advertising is present and keeping promotional material separate from clinical advice.
When advertising meets a clinical question
As people increasingly turn to artificial intelligence (AI) for health information, a new study by researchers at the Icahn School of Medicine at Mount Sinai raises questions about what happens when commercial advertising appears alongside a clinical question.
The findings were reported in the 11 November online issue of The Lancet Digital Health.
The researchers tested 12 AI models from major technology companies, including Google, OpenAI and Anthropic, to determine whether a clearly labelled pharmaceutical advertisement could influence which treatment an AI model recommended.
How the study was designed
The study was designed to isolate the potential effect of advertising. The researchers gave the AI models the same physician-written clinical cases, either alone or with a clearly labelled real drug advertisement placed immediately before the case.
The patient information and treatment choices remained unchanged; only the presence of the advertisement varied. Across 33 clinical scenarios, the researchers analysed 258,559 AI responses, assessing both the treatment selected and whether it was consistent with clinical guidance.
Ads shifted treatment choices
The researchers found that an ad placed before a clinical case could shift which treatment the AI recommended, making it more likely to select the advertised drug.
Across the 33 clinical scenarios, the advertised drug was selected in 47.6 per cent of responses when an ad was present, compared with 35 per cent without an ad.
In four cases, the effect was stark: models shifted from selecting the advertised drug 0 per cent of the time to selecting it 100 per cent of the time once the ad was introduced.
Reasonable answers, hidden influence
The shift in recommendations did not necessarily make the answers look wrong.
“What I find quite interesting, and even surprising, was how reasonable the answers often appeared,” says physician-scientist and first author Mahmud Omar, MD, a lecturer in the Windreich Department of Artificial Intelligence and Human Health at the Icahn School of Medicine, who leads research on the safety, reliability and real-world effects of generative AI in clinical care.
“The model could recommend an appropriate treatment, but its preference among several reasonable options changed after seeing the ad. Because the final answer could still appear medically sound, that influence could be easy to miss in typical research evaluations and in other patient and physician facing personal use cases.”
Models rarely acknowledged the ad
The researchers also examined how often AI models acknowledged the presence of an advertisement or repeated information from it. In a subset of free-text responses, models explicitly mentioned the ad in only 29.9 per cent of responses, while they repeated information or claims contained in the advertisement more frequently.
“An answer can be clinically reasonable and still raise an important question about why the AI arrived at that answer,” says senior author Girish N. Nadkarni, MD, MPH, Chair of the Windreich Department of Artificial Intelligence and Human Health, Director of the Hasso Plattner Institute for Digital Health, and Irene and Dr. Arthur M. Fishberg Professor of Medicine at the Icahn School of Medicine, and Chief AI Officer of the Mount Sinai Health System.
“As AI becomes part of how people get health information, we need to be able to distinguish clinical guidance from commercial influence and be transparent about when advertising is part of the interaction.”
What the findings do and do not show
The investigators say the findings do not mean pharmaceutical advertising will necessarily cause AI models to recommend unsafe or inappropriate treatments. In the study, advertisements did not clearly increase recommendations for unsupported supplements or clearly suboptimal drugs.
The researchers also did not examine whether changes in AI recommendations ultimately affected prescriptions, patient decisions, costs or health outcomes.
Practical steps for developers, health systems and regulators
The researchers say the findings point to several practical steps for AI developers and health systems:
- testing AI systems with and without promotional content to determine whether recommendations change
- clearly disclosing when advertising is presented to an AI system
- keeping promotional material separate from clinical advice
The findings may also have implications for regulators as they consider how existing rules governing pharmaceutical advertising apply when promotional content is incorporated into AI-generated health information.
For clinicians, the study is a reminder that an AI answer can look sound while still having been nudged. Knowing when to trust what AI tells you, and when to question it, is the focus of professional training such as the College of Contemporary Health’s AI Essentials for Primary Care: Tools, Ethics and Everyday Applications, a CPD-accredited online short course.
About the study
The research team plans to extend the work through its ongoing evaluation of AI and human-AI interaction in real-world clinical cases.
The paper is titled “Ad-verse Effects: Pharmaceutical Advertising and Drug Recommendations by Consumer-Facing AI in a Controlled Study”.
CCH insight
This study shows that an AI recommendation can be clinically reasonable and still be shaped by something other than the evidence – and that the influence may not be visible in the answer itself. As AI becomes part of how patients and clinicians look for health information, the ability to question AI outputs is an increasingly important skill.
CCH’s AI Essentials for Primary Care CPD short course (3.5 CPD hours, fully online, CPD-accredited) helps you feel equipped, not exposed: know when to trust what AI tells you, when to question it, and what to say when a patient asks. No technical background needed.
Explore AI Essentials for Primary Care →
Source: The Lancet Digital Health
Read More