Summary
Reader variability is the natural disagreement that appears when two qualified radiologists review the same scans, apply the same criteria, and still reach different conclusions about a participant’s disease. It is a well-documented and persistent source of difficulty in oncology imaging, and in a clinical trial, it can shape whether a therapy is judged to be working.
In an oncology trial, that judgment call, made repeatedly across hundreds of participants and many time points, becomes the raw material for endpoints like progression-free survival (PFS) and objective/overall response rate (ORR). It is tempting to treat these measurements as purely objective. In practice, two careful readers looking at the same images can land in different places, and that gap is what this series sets out to explain.
This article is part 1 of a series based on MERIT’s webinar, “Reader Variability in Oncology Clinical Trials: A Literature Review and Case Study.” Watch the full recording here.
How Do RECIST and RANO Standardize Tumor Assessment?
Response criteria such as RECIST, RANO, PCWG3/4, Lugano, and others exist to reduce this kind of noise. These standardized frameworks tell radiologists how to measure tumors, how to categorize change over time, and how to decide when a participant’s disease has progressed. The goal is to make sure a trial’s result reflects what the treatment did, not whoever happened to review the images.
Even with these rules in place, disagreement remains. That is not, by itself, a sign that someone made a mistake. Reader variability shows up consistently across cancer types, imaging modalities, and response frameworks, and researchers increasingly treat it as an expected feature of image-based response assessment rather than a flaw to be engineered away (Ford et al., J Clin Trials, 2016).
How Does Blinded Independent Central Review Work?
To limit bias from the physicians running a trial, the FDA recommends a process called Blinded Independent Central Review, or BICR, for pivotal oncology imaging studies. Images are sent to an independent core lab, where two readers evaluate them separately, each without knowing the participant’s treatment arm or the other reader’s conclusions.
When the two readers disagree, a third reader, called the adjudicator, steps in to resolve the disagreement. The share of participants who require this extra step is called the adjudication rate. It has become the standard real-world measure of reader variability, and the same underlying phenomenon is sometimes described as a discordance rate, or in its inverted form, as an agreement or concordance rate.
BICR is a meaningful safeguard against investigator bias. But as later data in this series will show, it does not eliminate disagreement between readers, and understanding why is the starting point for managing it well.
Why Does Reader Variability Matter?
Reader variability is not an abstract statistical curiosity. It shapes how a trial is designed, interpreted, and ultimately received by regulators, in four connected ways:
1. It affects endpoint integrity.
Progression-free survival depends entirely on when, and whether, a reader identifies disease progression, so two readers disagreeing on that call for the same participant produces two different possible outcomes for that participant.
2. It carries a statistical cost.
Disagreement between readers adds noise to the data, and that noise can reduce a study’s power to detect a real treatment effect, particularly in a trial built around a modest but clinically meaningful difference between arms.
3. It carries a cost and timeline burden.
Every adjudicated case requires bringing in a third radiologist, which adds expense and time to an already lengthy trial process.
4. It affects regulatory confidence.
Consistent, reproducible reads strengthen a submission’s credibility, while inconsistent ones can invite additional scrutiny and raise questions about a study’s conclusions.
How Common Is Reader Disagreement?
To put a number on it: a pooled analysis of 79 oncology trials found that adjudication was required in 42 percent of cases overall, with the rate varying by tumor type (Ford et al., 2016). Later work in lung and esophageal cancer trials has reported similar patterns, with adjudication rates commonly falling in a comparable range depending on the cancer type and trial design (Beaumont et al., 2021). In other words, disagreement between readers is not a rare edge case. It is a routine part of oncology imaging review, which is exactly why it deserves careful management rather than simply being treated as a footnote.
What’s Next in This Series
Taken together, endpoint integrity, statistical power, cost, and regulatory trust explain why reader variability deserves close attention. The next articles in this series will look at how common this disagreement is across large multi trial datasets, where it tends to originate, how it differs by cancer type and response framework, and what can actually be done to manage it.
This article is part 1 of a series based on MERIT’s webinar, “Reader Variability in Oncology Clinical Trials: A Literature Review and Case Study.” Watch the full recording here.
MERIT is a global clinical trial endpoint services provider specializing in the oncology, ophthalmology, and respiratory therapeutic areas, supporting sponsors with end-to-end central imaging review built around consistent, well-documented reader performance.
References
Ford R, O’Neal M, Moskowitz SC, Fraunberger J. Adjudication rates between readers in blinded independent central review of oncology studies. J Clin Trials. 2016;6:289.
Beaumont H, Iannessi A, Wang Y, et al. Blinded independent central review in new therapeutic lung cancer trials. Cancers. 2021;13(18):4533.
Iannessi A, et al. What are RECIST 1.1 progressions made of? Variability in double read oncology trials. Eur Radiol. 2026.