Interpreting Clinical Trial Data Is Not as Simple as It May Seem

Maurie Markman, MD

It is well understood that for a given clinical cancer trial to be labeled a success by the oncology community, multiple elements must be carefully and appropriately considered, undertaken, and completed. The research question must be clearly stated, with due thought given to whether the proposed study design will potentially provide an objectively valid answer depending on the results observed. For example, in the case of a randomized controlled study, this may be a statistically significant outcome. The protocol must be carefully written, and include unambiguous statements regarding eligibility and ineligibility criteria, drug/dose modifications for anticipated/unanticipated adverse effects, acceptable measures for defining disease progression and response, and the statistical methods to be employed to analyze study results.

Although study investigators can, and surely will, put their own spin on the interpretation of what the trial findings reveal, particularly in the setting of less-than-highly robust statistically significant results, well-established clinical trial rules will limit the conclusions that may be appropriately drawn. For example, if an investigator was evaluating the primary treatment of metastatic pancreas cancer or adjuvant therapy for triple-negative breast cancer, for one to declare that regimen A is clinically superior to regimen B in a given situation the 2 strategies must be directly compared in a properly designed randomized trial, the results of which must reveal a prospectively defined, statistically significant difference in a meaningful measure of clinical benefit (eg, progression-free survival, overall survival [OS], objective response rate).

An investigator, or other commentator discussing the study results, may certainly opine that the survival outcome observed in a population of patients treated with regimen A appears to be superior to prior phase 2 trials, previously published experiences, or retrospective reviews of institutional data with regimen B. But in many settings, such conclusions will not be considered truly definitive in the absence of direct comparative studies.

In contrast to the preceding general statements, the following commentary may be considered somewhat more controversial.

Consider, for example, the confusing public commentary that followed publication of the results of a randomized phase 3 trial that examined the clinical utility of screening colonoscopy in the prevention of colon cancer and the ultimate effect of this strategy on cancer mortality.¹ These findings, published in the New England Journal of Medicine, followed the clinical course of 84,585 study participants (age, 55-64 years) from Poland, Norway, and Sweden, 28,220 of whom were invited to undergo a single screening colonoscopy vs the remainder who did not receive such an invitation.¹

Based on a standard statistical intention-to-analysis (in this case, intention to screen), with a median follow-up of 10 years, although the risk of colorectal cancer was 18% lower in the invited group (0.98% vs 1.20%), the risk of death from colon cancer was not significantly different between those who received an invitation for a colonoscopy compared with no invitation (0.28% vs 0.31%; risk ratio [RR], 0.90; 95% CI, 0.64-1.16).¹ And when the risk of death from any cause was examined, the 2 groups were essentially indistinguishable (11.03% vs 11.04%; RR, 0.99; 95% CI, 0.96-1.04).¹

Some commentators claimed the absence of an impact on colon cancer— specific death and OS suggests the data failed to reveal a meaningful benefit associated with screening colonoscopy, a statement potentially difficult to dispute based on the intention-to-screen analysis. However, a far more appropriate interpretation of the data would be to recognize that the study objectively failed to address its primary outcome question because of the fact only 42% (11,843 of 28,220) of the invited group underwent the screening procedure.

Rather than showing that randomization to screening colonoscopy did not and does not impact colon cancer survival, this study’s data unfortunately— and rather strikingly—revealed the difficulty of establishing an effective large population-based colonoscopy screening program.

A second example of the potential dangers of inappropriate data interpretation with the risk of serious public health implications is provided by the recent report of the clinical benefit of mask wearing to prevent spreading respiratory viruses, including influenza and SARS-CoV-2.2 The authors examined findings from multiple published studies in this clinical arena. It is relevant to acknowledge that the Cochrane Library, under whose auspices this report was published, is known for the quality of its reviews, so when this report emerged, the results of which appeared to challenge the efficacy of masks, there was appropriate concern in the public health community.

For example, the report stated: “Wearing masks in the community probably makes little or no difference to the outcome of laboratory-confirmed influenza/SARS-CoV-2 compared [with] not wearing masks (RR, 1.01; 95% CI, 0.72-1.42; 6 trials, 13,919 participants; moderate-certainty evidence).”² In another section, the authors state: “The use of N95/P2 respirators compared [with] medical/surgical masks probably makes little or no difference for the objective and more precise outcome of laboratory-confirmed influenza infection (RR, 1.10; 95% CI, 0.90-1.34; 5 trials, 8407 participants; moderate-certainty evidence).”² And finally, they declare: “The pooled results of [randomized controlled trials] did not show a clear reduction in respiratory viral infection with the use of medical/surgical masks. There were no clear differences between the use of medical/surgical masks compared with N95/P2 respirators in health care workers when used in routine care to reduce respiratory viral infection.”²

However, the most important part of this report, which received far less press than the comments noted above, were the final conclusions: “…the high risk of bias in the trials, variation in outcome measurement, and relatively low adherence with the interventions during the studies hampers drawing firm conclusions.”² Would a more appropriate conclusion for this analysis have been that the available studies, despite the inclusion of relatively large sample sizes, were simply inadequately designed, undertaken, and/or analyzed to permit a meaningful conclusion?

In our fast-moving world where a report of a clinically important trial finding may be simply a 60-second sound bite for both lay and medical communities, it is critical that great care must be taken that reported conclusions are understandable to all, not just the statisticians.

Maurie Markman, MD, is president of Medicine & Science at City of Hope Atlanta, Chicago and Phoenix.

References

1. Bretthauer M, Loberg M, Wieszczy P, et al; NordICC Study Group. Effect of colonoscopy screening on risks of colorectal cancer and related death. N Engl J Med. 2022;387(17):1547-1556. doi:10.1056/NEJMoa2208375
2. Jefferson T, Dooley L, Ferroni E, et al. Physical interventions to interrupt or reduce the spread of respiratory viruses. Cochrane Database Syst Rev. 2023;1(1):CD006207. doi:10.1002/14651858.CD006207.pub6