hakule / iStock
By Jeffrey Brainard
For very busy working scientists, another invitation can be received from an academic journal for peer review. The work is time consuming, and rewards can seem intangible. What’s more, the judges work for free, even if the big commercial publishers that run many magazines make big profits.
But despite the occasional exuberant outcry of ‘I have to get paid for this’, scientists continued. Many mention duty to help advance their disciplines, as well as the need for reciprocity, because they know that other researchers voluntarily review their manuscript submissions.
But last week, researchers at a scientific publishing conference discussed a challenging question: should peer judges be paid?
The issue has attracted more attention as peer reviewers have become more difficult to recruit. Even before the COVID-19 pandemic began last year, which caused a snowstorm of submissions, magazines ‘exhaustion of judges’ were reported: In 2013, magazine editors had to invite an average of 1.9 judges to submit one complete review get; According to 2017, the number has risen to 2.4, according to a report by Publons, a company that monitors the work of peer reviewers.
During the debate at the Virtual Researcher to Reader conference, arguments were presented to paying judges by James Heathers, a former scientist in computational behavioral science who is currently the scientific head of a technology startup, Cipher Skin. Last year, Heathers attracted attention after publishing a manifesto, ‘The 450 Movement,’ arguing that $ 450 would be a reasonable fee for for-profit publishers to pay him per peer review. Heathers based the number in part on what business consultants in his field would recommend. Other judges may be able to negotiate lower amounts, he added.
Brad Fenwick, senior vice president at Taylor & Francis, a lucrative publisher with about 2,700 magazines, joined Heathers on the payback team. The couple claims that paying reviewers can correct a number of peer review deficiencies, including long delays in receiving reviews that too often have depth and content.
However, a payment team predicted serious consequences if $ 450 fees became the norm. Subscription costs will rise and unethical review may increase, said a team that includes Alison Mudditt, CEO of PLOS, the non-profit publisher of public articles, and Tim Vines, a publishing consultant and founder of DataSeer, a tool for sharing of data, include.
Here are excerpts from the debate for clarity and brevity. After the excerpts, you will find the results of recordings that determined on which side the audience was more convincing.
To pay cash against other rewards
Fenwick: Some editors are well compensated for their efforts. Why then should the same approach not be applied to peer reviewers? Universities offer faculties the freedom to supplement their income as paid consultants and / or by being involved in profitable enterprises. There is no reason why their contribution to the publishing industry should be treated less.
Heathers: Peer review is treated in the same way as any other general resource before it is regulated – air, water, land, etc. There is no downward pressure on the endless use of academic labor. And the easiest way to apply that pressure is to not appreciate the task. [only] with recognition, but with the traditional way of supporting skilled labor in every other industry, it’s money.
Mudditt: [Paid reviewing got] so little interest from researchers. A 2018 publons survey found that only 17% of respondents chose cash or payment in kind as something they would be more likely to accept review requests. [The top priority, selected by 45%, was more explicit recognition of the reviewing work from universities or employer.]
Vineyards: There are attempts to reward peer review, and it continues. Good judges become editors, their reputation in the community rises.
The question of whether magazines can afford to pay peer reviewers
AM: There is no practical way to pay judges without destroying peer review. Reviews vary greatly in length, quality and complexity. Where would we start assessing an appropriate fee? Why choose $ 450? There are some articles that are so complicated that maybe only a handful of experts on earth can review them. One major publisher tells me this [a fee of only $350] would wipe out the surplus they returned to society – no more investment in society’s research and researchers.
TV: An average of 2.2 reviews per article is very typical of magazines. And [assuming each reviewer is paid $450,] each revised article therefore costs $ 990. Of course, a cost for articles or article processing [required by some journals to make articles free to read on publication] is paid only by the articles accepted for publication, and the cost of reviewing the rejected articles is loaded into the APC. For a journal with an acceptance rate of 25%, they need to review four articles to find the article they accept. If we multiply by four, we get $ 3960 [to cover the costs of review for each accepted paper].
Imagine what it would look like in an entire industry. The additional cost to PLOS would increase $ 14 million per year, more than doubling the total annual expenses. The money would surely be better spent on the research itself and the solution to our most pressing global challenges.
BF: What can happen is that fewer papers are submitted because the cost goes up. And this is actually a big threat for publishers based on APCs who have to accept so many papers just to keep their cash flow stable.
[Paying reviewers is] a model to be tested. This is a value proposition: if it delivers higher quality publications and does it faster, what is its value to the scientific community? And for some [reviewers], the payment can be zero – they do not need it.
About contracts between judges and magazines
BF: A contract that offers an explicit exchange of value provides the necessary certainty around the time frame, the quality and the predictability of the review received.
AM: It is completely unrealistic to expect anyone to have the time or the expertise or the scale to manage and monitor hundreds of thousands of additional new contracts throughout the publishing system. Just will not happen.
JH: It’s very hard to believe that companies that download more than a billion a year manage [of scholarly articles] will somehow find it impossible to do contract and payment management.
The question of whether paying judges encourage unethical behavior
AM: In an APC publishing model, where reviewers have to be paid, even if they reject an article, but the magazine does not earn any revenue – as an editor, you want to find reviewers who will accept an article so that you can recover APC costs. The prospect of a quick price will also prompt some reviewers to comment on articles outside their areas, or to tell editors what they think they want to hear, so that they can be reappointed. … And just as we begin to see healthy and much-needed experimentation with new forms of reviews, such as pre-print reviews, they are likely to be killed.
JH: The idea that paid review is inevitably played out by dishonest judges in one way or another … assumes that an editor will hire their sight unseen … if he does not look at the work they have produced.
Prior to the debate, a scatter level among 64 audience members scored 41% for peer reviewers and 59% against it. Thereafter, the balance among 50 voters fluctuated to only 16% for payment, 84% versus.
Whether magazine editors across the board will remain equally pessimistic about payments remains to be seen. At the Taylor & Francis Group, a small variety of magazines focusing on pharmaceutical development paid peer reviewers for accelerated reviews, says Jennifer McMillan, director of communications. The publisher does not intend to increase the number of magazines paid by judges. Fenwick’s role in the debate was only to stimulate discussion, not to present company policy, she adds.