AI tool accurately predicts cancer patient outcomes based on pre-treatment data

Key Takeaways:

• A machine learning platform developed at Weill Cornell Medicine accurately grouped lung cancer patients by shared baseline traits and treatment outcomes using health record data.
• The model outperformed all previously published approaches in predicting treatment responses, using a real-world dataset of patients with advanced small-cell lung cancer.
• This tool could enhance both clinical trial design and individualised treatment planning, while offering insight into the biological underpinnings of patient subgroups.

Introduction: A Smarter Way to Predict Cancer Treatment Outcomes

A new artificial intelligence (AI) method developed by researchers at Weill Cornell Medicine and Regeneron Pharmaceuticals can accurately sort people living with cancer into groups with similar baseline characteristics and likely treatment outcomes. The breakthrough, published on 12 May in Nature Communications, could improve both patient selection in clinical trials and the tailoring of treatments to individual patients.

“We’re hopeful that this approach ultimately will be useful for testing and targeting treatments across a wide range of diseases,” said senior author Dr Fei Wang, founding director of the Institute of AI for Digital Health and professor of population health sciences at Weill Cornell Medicine.

Background: Addressing a Longstanding Challenge

Predicting which individuals are most likely to benefit from a particular cancer treatment has proven to be a major challenge for both pharmaceutical companies and healthcare professionals. Machine learning has long been viewed as a promising tool to uncover complex patterns in large datasets, such as those derived from electronic health records. However, traditional algorithms often fail to match these patterns to actual treatment outcomes.

“Groupings don’t always correspond closely to patients’ future treatment responses,” explained Dr Wang. Recognising this limitation, Regeneron scientist Dr Ying Li approached Dr Wang’s team with the aim of developing a more effective system.

“Our goal was to develop a platform that sorts patients with the target disease who are receiving the same treatment into groups sharing similar baseline characteristics and treatment outcomes,” said Dr Li. “We validated this method using a real-world database of advanced small-cell lung cancer patients treated with immune checkpoint inhibitors.”

Methodology: Training on Real-World Data

The machine learning platform was developed by first author Dr Weishen Pan, a postdoctoral research associate in Dr Wang’s lab. It was trained on anonymised health records from 3,225 individuals with lung cancer. Each record included 104 variables—ranging from blood test results and prescriptions to tumour stage and medical history.

The platform grouped patients into three distinct subpopulations:

• The group with the longest average survival time consisted mostly of women (55.5%) and had relatively low rates of co-occurring conditions such as diabetes and heart failure.
• The group with the shortest average survival time had less than half the mean survival of the first group, comprised mostly of men (66.2%), and exhibited more extensive tumour metastases along with blood test abnormalities suggesting liver, kidney, and inflammatory issues.

“Using a metric called the concordance index, we showed that the average performance of this new approach at predicting patient survival times was superior to that of standard statistical and machine learning methods,” said Dr Pan.

Validation and Broader Application

To test the robustness of their platform, the research team applied it to a separate dataset involving 1,441 people with non-small-cell lung cancer. Remarkably, the model generated nearly identical groupings in terms of both baseline traits and survival outcomes.

Dr Wang and Dr Li now plan further development and application of this approach, particularly for improving stratification in clinical trials of new cancer drugs. They also hope to use the tool to support more personalised treatment decisions in routine care.

Towards Understanding Disease Mechanisms

In addition to its immediate clinical utility, the model may also offer a new lens through which to explore cancer biology itself.

“We’ll probably need more than electronic health record data for this, but we do want to understand the biological mechanisms that explain these distinct patient subgroups,” said Dr Wang.

The research marks a significant step forward in the use of AI to enhance precision oncology, offering the promise of more targeted therapies and improved outcomes for people living with cancer.