E. Petersen /Science
By Charles Piller
This story was supported by the Science Investigative Reporting Fund. Help please Science to pursue ambitious journalism projects.
In June 2020, in the biggest pandemic research scandal to date, two of the major medical journals each pulled a sensational study of COVID-19 patients. Thousands of news articles, tweets and scientific commentary highlighted the scandal, but many researchers apparently did not notice it. In a survey of the most recent 200 academic articles published in 2020, with reference to the papers, Science found that more than half – including many in leading journals – used the shameful papers to support scientific findings, and did not take note of the withdrawals.
COVID-19 ‘is such a popular topic that publishers are willing to publish without checking,’ even if the withdrawals have made headlines worldwide, says Elizabeth Suelzer, a reference librarian at the Medical College of Wisconsin, who wrote about problematic quotes after a withdrawn study in 1998 in The Lancet the vaccination falsely linked to autism.
Both the withdrawn COVID-19 papers, one in The New England Journal of Medicine (NEJM) and the others in The Lancet, was based on a large database of patient records compiled by hospitals worldwide by Surgisphere, a small business run by vascular surgeon Sapan Desai, who co-authored each article. The 22 May 2020 Lancet paper has apparently shown that hydroxychloroquine, an antimalarial drug promoted by President Donald Trump and others, can harm COVID-19 patients rather than. Its publication led to a temporary halt in a major clinical trial and sparked an already divisive debate over the drug, which apparently was no help against COVID-19. May 1 NEJM article confirms other evidence that people who are already using certain medications for blood pressure do not face a greater risk of death if they develop COVID-19.
However, questions quickly arose about the validity and even existence of the Surgisphere database, and the withdrawals followed on 4 June. But of the 200 papers submitted by Science—Everything published after the retreats – cited one of the shameful studies inappropriately. In several cases, it was a primary source for a meta-analysis that combined multiple studies to draw overarching conclusions. In most, the studies were cited as scientific support or context. Science also found a handful of articles that critically cited an influential April pre-print based on the same Surgisphere dataset, describing the antiparasitic agent ivermectin as beneficial in critical COVID-19 cases. (However, there is no standard way to retrieve preprints.)
Ivan Oransky, co-founder of the website Retraction Watch, says such mistakes occur because “people do not check references intentionally or negligently.” Many authors copy and paste lists of seemingly relevant quotes from similar papers without actually reading them, he says. “It’s scary. This is terrible, but common. ‘
Many of the infected quotes appear in articles published by few well-known magazines, but at least a dozen have found their way to major publications. For example, three articles in PLOS ONE, the prominent open-access journal, cited the withdrawn papers in discussions on pandemic conditions in Europe. N 28 December paper in the Proceedings of the National Academy of Sciences (PNAS) – one of the most influential journals – discussed the risks and benefits of COVID-19 medicines and noted the Lancet withdrawn in his quotations, but in the text the hydroxychloroquine findings of the newspaper are merely mentioned as ‘controversial’.
Related
Editors of the two publications said they would correct the references and take steps to prevent such problems in the future. Renee Hoch, a PLOS ONE editor and publication ethics manager, wrote in an email that the publication relies on authors and outside, volunteer editors to check quotes, and she was surprised when she Science. “We are currently pursuing this issue with high priority in the light of the implications for public health and ongoing research on COVID-19,” she wrote.
Hoch added that reliance on repatriation, “directly or in the form of supporting references,” can be harmful. ‘[W]here the withdrawn work has clinical implications, it can result in direct risks for patients. ‘
In a written answer to questions about the citation of the Lancet article, May Berenbaum, Editor-in-Chief of PNAS, said: “The authors should have actually removed the citation, added more text as to why they included it, or quoted the revocation notice themselves.” Since no editor or reviewer experienced the problem, she said, “I plan to discuss with staff the inclusion of such performance in the processing of manuscripts.” A co-author of the article, biostatistician Clelia Di Serio of Vita-Salute San Raffaele University, later said the reference to Lancet paper would be erased.
Some magazine editors have noted encouraging factors. In Stroke, a leading medical journal, referred to an article in December on ischemic stroke in COVID-19 patients referring to the NEJM paper without mentioning the withdrawal. Stroke Editor-in-Chief Ralph Sacco wrote in an email that the withdrawal took place after the article was initially received. However, according to the newspaper itself, a review of the article was resubmitted months after the event. Sacco said he would not make any correction because the withdrawal “is not material to the findings.”
A December 16 paper on SARS-CoV-2 genetics in Nature communication, another high-profile journal, also has the NEJM article without reference to the withdrawal. Elisa De Ranieri, the editor-in-chief of the magazine, tells Science her journal does not regularly check for ‘withdrawals or other deferral updates’. A lead author of the paper, biomathologist Maik Pietzner at the University of Cambridge, said that although the article was submitted after the withdrawal took place, it was written in advance, and that the current pandemic responds immediately. ‘ However, the paper was published 4 months after its submission.
Suelzer says inappropriate quotes from withdrawn articles are hard to excuse. Retraction Watch publishes a free withdrawal database integrated into a number of automated quotes checking services, including scite.ai, Zotero and RedacTek. The failure to use such tools is a disgrace to readers and researchers, ‘says Suelzer. “These are pretty low bars.”
Yet Oransky estimates that up to 90% of the quotes in withdrawn articles in biomedicine do not mention the fall of grace. ‘Half the time [as seen with the Surgisphere papers] is an improvement. It’s shocking. ”