Peer-Reviewed Data Present an Essential But Problematic Component of Oncology Research

Maurie Markman, MD

After several years' delay, the eagerly awaited peer-reviewed data from 2 randomized phase 3 trials of aducanumab (Aduhelm) were published.^1,2 The FDA’s approval in 2021 of Biogen’s Alzheimer disease treatment has been highly controversial. Considering the importance of this information and the societal implications associated with the widespread use of the expensive medication, one might have anticipated that the results would be published in a high-impact medical journal. This would, hopefully, ensure the adequacy of essential scientific peer review by recognized experts, including clinicians, laboratory scientists, clinical trialists, and statisticians in the field.

Instead, outcomes were remarkably published in the rather low-profile Journal of Prevention of Alzheimer’s Disease.² Of interest, the second author on the aducanumab manuscript is the editor in chief of the journal and a consultant for the drug’s manufacturer. What do these observations potentially say about the adequacy of peer review?

A Tangled Web

There are several critical components associated with the formal evaluation and ultimate acceptance of proposed new information within the scientific domain. However, none is arguably more essential than that of the conduct of quality peer review. One frequently hears that a particular manuscript or scientific claim has “yet to undergo peer review.” The implications may be that until this process has been completed and the article is published in a reputable peer- reviewed journal the effort remains a work in progress and any suggested assertions may be taken as preliminary communications.

Unfortunately, as with most overly simplistic statements related to the scientific process, there are serious concerns about textbook descriptions of peer review specifically within the realm of clinical research. In the opinion of this commentator, these matters pose a threat to both the objectivity of the field of investigative medicine and, perhaps most importantly, to the public’s support of clinical science.

Some of these issues have been discussed in the academic literature but have found little interest in the lay media. These include large numbers of what have been labeled predatory journals where, for a price, almost any manuscript can be published with either no peer review or a blatantly fraudulent review process.³

Perhaps more importantly, who is being asked to provide peer review and what is the objective quality of their reviews? With the explosion in the number of clinically related journals, there is an ever-increasing need to find reviewers who are both interested in and knowledgeable about a particular subject. Accounting for increasing specialization and complexity of translational and clinically related laboratory research in almost all fields of medicine, it may be difficult to maintain the essential supply of objectively appropriate and knowledgeable reviewers.

Further, although the honor associated with being asked to provide peer review for a high-impact journal will likely be sufficient to entice otherwise busy individuals to provide the time and effort required to produce a meaningful review (without financial compensation), this may not be the case for a large percentage of other journals seeking the services of these same individuals. As a result, it is relevant to inquire what this complex state of affairs says about the overall quality of peer review in medical journals.

There has been considerable discussion regarding conflict of interest by investigators in their dealings with study sponsors and the effect of these relationships on the objectivity of submitted manuscripts. However, it is rare to hear concerns about the potential conflict a peer reviewer or journal editor may present on a subject. The issue being highlighted here is not the existing relationships with entities that would be declared in the same manner as required by those submitting the manuscript, but rather the more subtle favorable or unfavorable biases that may seriously interfere with objective peer review.

Calling Review Selection Into Question

The potential for serious subjective, underappreciated bias and lack of scientific rigor associated with peer review in academic oncology is inevitable and may call certain manuscripts into scrutiny.

For example, is it appropriate to question the bias of results of a peer-reviewed manuscript published in a gynecologic oncology specialty journal that conclude that the care of patients with advanced ovarian cancer provided by a gynecologic oncology specialists is associated with improved overall survival compared with outcomes when treated by medical oncologists?⁴ Similarly, should results of a paper published in a journal representing large comprehensive cancer centers that conclude that patients who underwent surgery at high-volume facilities “consistently had improved overall survival compared with those at low-volume centers” be questioned?⁵

The use of the preceding examples is not to suggest specific defects in the conclusions. There are published results that suggest a much more concerning bias. I conclude this column with the example of conclusions drawn by authors that should have been caught by peer reviewers and journal editors prior to publication. The paper in question examined FDA approvals of cancer drugs between May 1, 2016, and May 31, 2021.⁶ The authors found that more drugs were approved to be used as later lines of therapy rather than to displace existing therapies. They concluded that “these later-line drugs may benefit patients with few alternatives but add to the cost of care because competition in the drug market is a key factor in leading to lower drug prices.”⁶

Although a potentially valid statement, it would have been highly relevant to also acknowledge in the publication the meaningful effect of later lines of therapy on patient survival and quality of life. Further, the authors could have cited data supporting the suggestion that many advanced cancers are increasingly becoming more chronic disease processes. Who were the peer reviewers and journal editors who let this critical detail slip through the cracks?

References

1. Alzheimer’s drug data out at last. Science. 2022;375(6587):1323. doi:10.1126/science.abq1701
2. Haeberlein SB, Aisen PS, Barkhof F, et al. Two randomized phase 3 studies of aducanumab in early Alzheimer’s disease. J Prev Alzheimers Dis. 2022;9(2):197-210. doi:10.14283/jpad.2022.30
3. Moher D, Moher E. Stop predatory publishers now: act collaboratively. Ann Intern Med. 2016;164(9):616617. doi:10.7326/M15-3015
4. Ko EM, Bekelman JE, Hicks-Courant K, Brensinger CM, Kanter GP. Association of gynecologic oncology versus medical oncology specialty with survival, utilization, and spending for treatment of gynecologic cancers. Gynecol Oncol. 2022;164(2):295-303. doi:10.1016/j.ygyno.2021.12.001
5. Stoltzfus KC, Shen B, Tchelebi L, et al. Impact of facility surgical volume on survival in patients with cancer. J Natl Compr Canc Netw. 2021;19(5):495-503. doi:10.6004/jnccn.2020.7644
6. Benjamin DJ, Xu A, Lythgoe MP, Prasad V. Cancer drug approvals that displaced existing standard-of-care therapies 2016-2021. JAMA Netw Open. 2022;5(3):e222265. doi:10.1001/jamanetworkopen.2022.2265