How Science Feedback flunked Fact-Checking 101

Note: See statement of transparency following the article.

Science Feedback’s fact check of a pro-life video used poor methods and the IFCN’s review missed it

An August 30, 2019 Science Feedback fact check (updated Sept. 3, 2019) ruled “Inaccurate” pro-life advocate Lila Rose’s claim “Abortion is not medically necessary.”

Science Feedback’s status as a Facebook fact-checking partner, made possible through its commitment to the International Fact-Checking Network’s statement of principles, allowed Facebook to use the fact check to justify limiting the distribution of the Rose video.

Young America’s Foundation posted and distributed the video on Facebook. When Facebook slapped restrictions on the video, Young America’s Foundation objected, saying Science Feedback’s fact check was biased. The IFCN used an external assessor to review the case, and Facebook suspended suggested viewing of the fact check to Facebook users wanting to watch the Rose video. The IFCN eventually ruled the fact check accurate while saying Science Feedback failed to properly uphold IFCN standards by not disclosing the advocacy positions of the consulted experts.

We will argue Science Feedback fact-checked poorly and the IFCN review overlooked the biggest problems with the fact check.

Fact-Checking 101

One of the most basic responsibilities of the fact checker consists of presenting claims in context.

Science Feedback failed that requirement spectacularly with its fact check of the Rose video (transcript here):

This video, which has been shared more than 6,000 times and is currently trending on Facebook, records part of a speech by Lila Rose at a Young America’s Foundation event. In this video, she claims that “abortion is not medically necessary”.

Physicians who evaluated this claim found it to be inaccurate.

Science Feedback does not tell us whether those physicians were permitted to see Rose’s claim in context.

And it gets worse.

Tangled up in Euphemism

Science Feedback faulted Rose for trying to redefine abortion. Rose said abortion is “the direct and intentional destruction of an innocent human life,” distinguishing that from cases where early removal of a fetus results in the death of the fetus in spite of efforts to keep it alive.

Science Feedback insisted on a different definition:

(A)bortion is medically defined as a procedure to end a pregnancy – this definition does not change depending on the reasons for an abortion, i.e. whether the procedure is motivated by an unwanted pregnancy or medical emergency or some other situation has no effect on its medical definition.

We judge that the use of euphemism in medical terminology clouded Rose’s point for the fact checkers at Science Feedback. After all, inducing labor is a procedure used to end a pregnancy. So is caesarean section. But we do not consider either of those procedures abortions because doctors tend to use those procedures to deliver living babies. So “end a pregnancy” does not simply mean “end a pregnancy.” It serves as a euphemism for terminating the fetus. Or killing the fetus, if we’re trying to avoid the hint of yet more euphemism.

Of note, Science Feedback’s chosen definition of “abortion” specifies removal of the fetus “from the uterus,” apparently complicating Science Feedback’s claim that doctors use abortion to treat ectopic pregnancies.

Science Feedback used word games to obscure Rose’s reasonable point: A useful and reasonable distinction may be made between a procedure intended to kill a fetus and a procedure not intended to kill the fetus.

In context, Rose made that point clear. Science Feedback’s suggestion that Rose’s rhetoric amounted to a “No true Scotsman” fallacy holds no water given the relatively obvious principle behind the distinction she drew.

What Did Rose Say that was Wrong?

Science Feedback said Rose was wrong for saying abortion is not medically necessary. We’ve seen that Science Feedback used equivocation (abortion is supposedly any procedure used to end pregnancy regardless of whether doctors try to end the life of the fetus) to denigrate Rose’s point.

Did Rose say anything that was wrong?

Science Feedback also charged that Rose misled about the possibility of delivering early in the case of certain problem pregnancies such as pre-eclampsia (bold emphasis added):

Lila Rose’s further comments suggesting that physicians “could do an early delivery if [the mother] is experiencing pre-eclampsia or she has a very severe condition that you need to deliver that baby early” instead of performing abortion also lacks context so as to be misleading. In order for early delivery to save both the life of mother and child, the fetus first needs to have developed sufficiently that it stands a chance of surviving outside of the womb. This caveat is not mentioned in the video. The comments do not take into account the fact that a fetus may not have reached a sufficiently mature gestational age during a maternal emergency – in such cases, abortion would be the fastest and safest way to terminate the pregnancy.

Science Feedback perhaps makes an even worse case here than for its first attempt to fault Rose.

Most glaringly, Science Feedback misquoted Rose by leaving out an important “perhaps.” Compare Science Feedback’s version of the quotation with what Rose said (bold emphasis added):

Now, you could perhaps do an early delivery if she’s experiencing pre-eclampsia or she has a very severe condition that you need to deliver that baby early, but in that situation you don’t go in there with a needle or forceps to destroy that baby before birth.

If Rose’s “perhaps” occurred before “could” then we can see cutting Science Feedback a break. But we cannot excuse a fact checker for simply omitting an important word from the midst of a key quotation where it materially affects the meaning.

Using the correct quotation does much to diminish the appearance that Rose was suggesting early delivery as an automatic fix for complications in early pregnancy. And Rose revisited the challenges of assisting the survival of early deliveries later in her speech, albeit without receiving any credit for it from Science Feedback.

Is it Medically Necessary to Directly Kill a Fetus?

Was Rose right that directly killing a fetus is not medically necessary?

If only Science Feedback had bothered to address Rose’s claim in context!

We detect no evidence in the Science Feedback fact check that would contradict Rose’s assertion. Science Feedback consistently referred to abortions within the definition it preferred, except for that pesky restriction to uterine pregnancies, saying abortion means any termination of pregnancy regardless of purpose or the survival of the fetus.

It did not address Rose’s claim as the context would have suggested, except to dubiously suggest Rose had redefined “abortion” to make it medically unnecessary by definition.

We don’t count the effort to introduce the health insurance concept of medical necessity as an answer to the question. That notion has to do with customary medical practice and whether insurance will pay for it, not whether a medical procedure is literally necessary in the therapeutic sense.

It’s clear that’s not the kind of medical necessity Rose was talking about.

We’re left to wonder why Science Feedback did not bother to fact check the gist of Rose’s claim.

Questioning the IFCN Judgment

We found failure to consider context, inaccurate quotations and an argument founded on ambiguity in Science Feedback’s fact check.

What did the International Fact-Checking Network find with its investigation?

The findings of Science Feedback’s fact-check were based on publicly available scientific evidence and as not the result of any bias. The claim that “abortion is never medically necessary” is false and inaccurate.

All of the publicly available scientific evidence Science Feedback introduced applied to cases where the aim of the procedure was not designed to bring about the death of the fetus and as such ignored the context of Rose’s claim.

The process used by Science Feedback to select the original claim to review was sound and not the result of any systemic bias, and a review of the 10 last fact-checks indicates no systemic bias in the selection of claims to check.

As a practical matter, claim selection is inevitably subject to selection bias. We’re not sure what distinction the IFCN tries to draw by using “systemic” as a modifier. We doubt the IFCN examination ruled out systemic bias however the IFCN defines it. We think it more accurate to say the IFCN did not detect systemic bias, albeit that wording perhaps makes too clear the relative emptiness of its assurance.

The failure to declare to their readers that two individuals who assisted Science Feedback, not in writing the fact-check but in reviewing the evidence, had positions within advocacy organizations, and the failure to clarify their role to readers, fell short of the standards required of IFCN signatories. This has been communicated to Science Feedback.

We agree with the IFCN that Science Feedback should have disclosed the advocacy positions of its science advisors.

But that was not the chief problem with the Science Feedback fact check. It ignored context, altered a quotation and ultimately relied on an equivocal argument.

The presence of those problems suggests bias at Science Feedback regardless of story selection bias.

Rose made the point that her group opposes abortion understood as the direct and deliberate killing of a fetus. Science Feedback clumsily twisted that into an attempt to confuse people about the definition of abortion.

The fact check was conducted improperly and Facebook should not use it to limit distribution of the Rose video.

Instead, Facebook will likely go back to using it to suppress reasonable political speech after Science Feedback applies a Band-Aid of disclosure to its fact check.

Update Oct. 5, 2019: We noticed pretty quickly that Science Feedback has its Rose fact check under two different domains. But it took until today for us to notice that Science Feedback has not consistently updated the version hosted under the sciencefeedback.co domain. The most recent update for that version (archived today) was Sept. 3, 2019. The most recent update for the healthfeedback.org domain was Oct. 1, 2019 (archived today). Neither version shows the needed corrections we’ve pointed out above.

Disclosure: Authorial Bias at Zebra Fact Check

We part with the IFCN on the issue of fact checker bias.

Zebra Fact Check was founded to help improve fact-checking, starting with abandonment of the facade of personal or organizational objectivity favored by mainstream U.S. fact checkers. The author, Bryan W. White, is a conservative and sympathetic to pro-life arguments. But we hold that disclosing bias makes it okay to fact check virtually any topic, with the commitment to present a fact check free of significant error and with every attempt to avoid misleading readers.

Each page at Zebra Fact Check features a “Report an Error” button. Where we make a mistake, let us know.

Update July 15, 2022: Added a second link to our Lila Rose transcript to make clear the origin of a quotation.