Over the past few months, Facebook has used third-party fact investigators to decide which COVID-19 news reports and options are false or misleading. Recently, one determined that a Wall Street Journal Opinion article by Marty Makary, MD, MPH, Johns Hopkins Professor and Editor-in-Chief at MedPage Today, was misleading. Specifically, “Three scientists analyzed the article and underestimated its overall scientific credibility,” according to HealthFeedback.org, Facebook’s selector in this case.
Makary argues in his open that COVID-19 mostly disappears in April. I want to make it clear: I’m not interested in arguing Makary’s opinion, and that’s what a lawyer is saying. April 30 will be here soon, and we’ll find out. Instead, I’m interested in thinking about how an $ 800 billion business can decide when an open by a professor can be ‘misleading’.
How does this third-party factual system work? Does it ask for reviews from the best academics in a field or discipline? Why does it focus on an op-ed? When I researched this piece, I discovered that the process is obscure. What was clear, however, was who the judges were.
They are excessively academics on Twitter who count many followers. They mostly have similar worldviews and advertise the views on Twitter. In another case, a reviewer had already tweeted criticism of the article before being selected as a fact checker. It’s not an independent or fair process – it’s criticism of Twitter celebrities to quench differing opinions.
How it works
Facebook has asked HealthFeedback.org to review at least some articles. The site appears to be asking between two and four reviewers per article, compiling their feedback before giving a verdict. The site appears to be new. It contains three reviews of articles before COVID-19 and seven thereafter. There have been ten reviews before COVID-19 and 19 since January 2020. I extracted data on all reviews.
Fact checkers or Twitter celebrities?
Of the 19 fact-checkers selected since the inception of COVID-19, 15 (79%) have active Twitter accounts. These people are followed by an average of 42,000 followers (median 10,000). Four of the fact checkers have served more than once.
Let’s compare the reviewers of COVID-19 with fact checkers before the COVID-19 and academics in general. Among the ten fact checkers for three reviews on HealthFeedback.org before COVID-19, there were no recurring reviews. Half were on Twitter and their average number of followers was 442 (median 130).
To compare these rates with the average academic, I visited the faculty of Johns Hopkins University’s Department of Epidemiology. I selected ten academics and searched for their Twitter accounts. Only three out of ten had Twitter accounts with an average following of 800 (median 120).
This should come as no surprise: the average scientist is not on Twitter, including many of the best and brightest. Yet HealthFeedback.org looks for the fact-checking of pandemics from Twitter celebrities who have an average of 40,000 followers.
The independent fact checkers used on this site appear to be much more active than the average member of the epidemiology on Twitter. Several have expressed their views on COVID-19 policy and support for continued restrictions, while Makary suggests that we intend to mitigate restrictions.
No matter how someone feels about who is right – is it a fair process? They review an opinion article using scientists who are disproportionate to a website that enables them to advertise their opinion in advance. I promise you, I can search on Twitter and find three people who will review Makary positively and three who will review him negatively just by reading the tweets of the past.
A conflict of interest
In one case, it appears that the fact checker was chosen because he had already written a Twitter thread that was critical of the article. The review notes that it was ‘lightly processed for clarity’. In other words, a Twitter user announced that he was critical of an article, and then their view was selected as independent fact checking. This is a very problematic selection process, as independence usually implies that a reviewer has not pre-judged the material.
This also apparently contradicts the process where the editor selects the article before selecting reviewers. Imagine that a juror was chosen at a trial because they had previously tweeted, or they were sure, that the accused was guilty.
It all starts and ends on Twitter
The fact-checking summaries show a greater debt to Twitter. The article that the Wall Street Journal op-ed by Makary also includes snapshots of tweets by Eric Topol, MD, and Caitlin Rivers, PhD, to bolster his claim. Aside from the huge reliance on academics who happen to be on Twitter as fact-checkers, it seems that the editors are following the comments on Twitter closely to guide them, perhaps even to which topics to choose.
This is not independent fact checking, but group thinking
Compiled, the fact check is suspicious. It turns out to be a website with its own policy ideas on COVID-19, which select academics who are popular on Twitter and have stated a position – and give them the chance to dull ideas that are against their own.
It feels like a high school clique. These are the popular kids. They use their position to label views with which they do not agree as ‘misleading’.
Some think it’s just a battle of ideas, but just a debate. There would be debates if these judges would write a rebuttal, but they go further. They use the brute force of the platform to literally describe Makary’s opinion as misleading. This is a big leap out of the debate. Labeling uses the reader immediately of their ability to decide for themselves. It is antithetical to the spirit of academia.
Facebook is a sea of garbage and that’s what they’re policing?
I have to state the obvious: Facebook is a sea of garbage. Illogical arguments, false allegations, harmful views – you can open Facebook and find what offensive idea you want. That’s why I do not open it. And yet there is only a small subset of stories that the organization wants to label ‘misleading’. And one of them is an essay by Makary? This whole scenario is bizarre!
Meanwhile, there are news reports that are clearly wrong and for which Facebook is not acting.
What I did not find
I found no explanation as to which of the billions of articles and Facebook posts the fact checkers are being asked to review. How to choose the editor? I found only a vague explanation of the standard for judging an article, not how disputes among judges are handled. I did not find an explanation as to how the judges are selected and what happens if they refuse. I did not find information on who and who is paying. I have not found evidence of how appeals are handled.
This is an op-ed
I must mention again the Kafkaesque point that Makary’s piece is an op-ed, which by definition is his perspective based on his interpretation of data. Like a Franz Kafka novel, Makary got a show trial through Facebook and is found guilty of misleading the public with his opinion piece by a bunch of Twitter celebrities, whose opinion is well known. I do not know if Makary is right or wrong, and time will tell, but what I do know is that this process is not acceptable or fair. Facebook should not call it ‘fact checking’. It is not right. It’s just to check out what their friends have to say on Twitter.
Vinay Prasad, MD, MPH, is a hematologist-oncologist and associate professor of medicine at the University of California, San Francisco, and author of Malignant: How Bad Policy and Bad Evidence Harm People with Cancer.