Why Fact Checkers Need a Watchdog

Who fact checks the fact checkers?

Since it started publishing occasional fact checks in 2010, Zebra Fact Check has found quite a few significant errors by mainstream media fact checkers. Unfortunately, our criticisms only rarely result in adequate changes in the original mainstream media fact checks. We held out some hope that the International Fact-Checking Network, hosted by the Poynter Institute (owner of PolitiFact), might work to hold fact-checking organizations accountable. But the IFCN spends so much time promoting fact-checking that it seems to have little time for activities that might make some fact checkers lose credibility in the short term.

We see a need for a watchdog organization that holds the media, including and especially fact checkers, to account. So that’s what we’re working toward.

In the remainder of this post we present a case study showing the need for that organization.


Trump on Civilians with Guns: The PolitiFact Video

On March 12, the blog PolitiFact Bias published a post of mine partly dedicated to criticizing a PolitiFact fact check (the other part criticized a “criticism” of that fact check that instead praised PolitiFact’s work while ignoring its errors). President Donald Trump tweeted that the presence of a trained civilian with a gun in the room with shooter Omar Mateen might have stopped or at least lowered the casualty count from the 2016 Pulse nightclub mass shooting in Orlando.

PolitiFact rated Trump’s statement “False.”

PolitiFact reported that an Orlando-area police officer had worked security “at the club” and exchanged fire with Mateen. PolitiFact quoted from a government report passages indicating that the guard exchanged fire with Mateen but from the parking lot, not from inside the club.

PolitiFact either reasoned that the Pulse parking lot counts as “within the Orlando club” or else that Trump’s failure to mention the guard in the parking lot somehow made Trump’s reasonable statement false.

We asked PolitiFact to either correct its fact check or defend its reasoning from our criticism.

PolitiFact did neither.

Then we noticed PolitiFact created a video version of the fact check. The video version likewise promotes the misleading impression that an armed guard inside the Pulse nightclub exchanged gunfire with Mateen.
We used the script provided to embed the PolitiFact video. Perhaps it will work.


PolitiFact’s Deception, Explained Step by Step

The following images from PolitiFact.com, asserting fair use for purposes of media criticism.


“President Donald Trump recently argued that armed citizens can prevent mass shootings…”

Trump’s argument was a matter-of-fact tweet, at least in the form PolitiFact used in its fact check.

Is there any question that armed citizens can prevent mass shootings? We think the premise serves as part of the justification for armed guards such as the one in the Pulse parking lot. Why do armed guards exist? Why do we tend to equip police officers with guns? Armed guards improve the public perception of safety.



“and cited the 2016 shooting rampage at Pulse nightclub in Orlando.”

Trump did not cite the Pulse nightclub shooting as an example of a mass shooting prevented by a civilian with a gun. Trump cited it as an example where a civilian (in the room) with a gun might have prevented the mass shooting.



“‘If you had one person in that room that could carry a gun and knew how to use it, it wouldn’t have happened,’ Trump said Feb. 28, 2018.”

PolitiFact trimmed the quotation for the sake of its video. Trump said (bold emphasis added) “If you had one person in that room that could carry a gun and knew how to use it, it wouldn’t have happened, or certainly to the extent that it did.”  Is it okay to leave out information that may change the impression left on the audience?



“But that ignores a crucial fact about how the attack unfolded.”

What makes the omission a “crucial” one? Watch for PolitiFact’s explanation.



“An armed off-duty police officer was working security that night.”

PolitiFact explained what it counts as a “crucial” omission but not why it counts the omission as crucial.

Meanwhile, PolitiFact’s video account omits the fact that the off-duty police officer, Adam Gruler, was working security in the Pulse nightclub’s parking lot. Gruler was not in the room with Mateen. Given that Trump specified that a person “in that room” and “within the Orlando club” with Mateen might have thwarted the shooting, we have reason for considering PolitiFact’s omission crucial.

PolitiFact’s written fact check at first led us to believe Mateen was confronted by an armed guard inside the Pulse nightclub. Why? Because it failed to emphasize the location of the guard, and because the “False” conclusion only follows if reality had contradicted Trump’s “in that room” phrasing. PolitiFact’s video leaves an even stronger false impression that the guard exchanged fire with Mateen while inside the club with him.



“The officer exchanged gunfire with the shooter two separate times.”

First, note that PolitiFact still has not explained why it is crucial that Trump omitted mention of the security guard in the parking lot who fired at Mateen.

Second, we charge PolitiFact with potentially exaggerating its report. The two detailed government reports on the Orlando club shooting say the off-duty officer fired shots at Mateen on two occasions. It has no reports of an “exchange” of fire where the officer and Mateen fired at each other.

The U.S. Department of Justice published a report stating:

At 2:02 a.m., still within the first minute of the attack, one of the three DJs in the club turned his music down and listened closer. With the volume lower and the sounds no longer matching thebeat of the music, it was clear that it was not a special effect but something much more real. The DJ turned the music off and yelled, “Run! Get out! There’s a guy with a gun!”

Outside, in the Pulse parking lot, an Orlando Police Department (OPD) detective who was working extra duty at the club—to provide outside security and to provide assistance to security personnel inside the club if needed—heard the shots that were being fired; at 2:02:17 a.m., he broadcast over the radio, “Shots fired, shots fired, shots fired,” and requested additional officers to respond. The detective told the assessment team that he immediately recognized that his Sig Sauer P226 9mm handgun (shown on page 17) was no match for the .223 caliber rifle being fired inside the club and moved to a position that afforded him more cover in the parking lot. Two patrons attempted to flee through an emergency exit on the south side of the club. When the detective saw the suspect shoot them, he fired at the suspect.

The report later picks up from that incident:

After the detective fired at the suspect, the suspect doubled back into the club and walked deeper into the Jewel Box section. He made his way through the main dance area, indiscriminately firing his weapons (a rifle and handgun). As the suspect traversed the club, at 2:05 a.m., the OPD detective engaged him again, firing from the parking lot into the club.

The Orlando Police Department’s account of the shooting likewise fails to mention Mateen shooting at the parking lot guard (Gruler).

An earlier report from the OPD says its officer (Gruler) “engaged in a gun battle” with the suspect (Mateen) but offers no details.

PolitiFact cited reports from The New York Times and Orlando Sentinel accounts saying Gruler exchanged fire with Mateen, but the reports do not specifically identify the source(s) of the information.

Did Mateen fire at Gruler? We think if Mateen fired at Gruler the detailed report from the U.S. Department of Justice would likely have mentioned it. We contacted the Orlando Police Department asking for evidence, including specific eyewitness accounts, that Mateen fired at Gruler. We will update this item if we receive any reply.



“The shooter was eventually slain after a three-hour standoff. But he managed to kill 49 people in the attack.”

We find no problem with these two statements, though neither explains PolitiFact’s notion that Trump made a “crucial” omission by failing to say an armed security officer was in the parking lot.



“We rated Trump’s claim False.”

PolitiFact’s video faults Trump for omitting “crucial” information but itself omits the fact that Detective Gruler was not in the room but instead in the Pulse nightclub parking lot. Trump had specified that an armed person in the same room might have prevented or reduced the death toll. Moreover, the fact check never explains what was supposedly false about Trump’s statement.

PolitiFact never produces any coherent grounds for its “False” rating of Trump’s statement.


Needed: A  New Kind of Media Watchdog

We need to hold fact-checkers accountable for spreading misinformation.

PolitiFact made a plain mistake. We pointed out the mistake to PolitiFact and to the director of the International Fact-Checking Network. The IFCN stands as the closest thing to a fact checker accountability system.

Nothing about the story changed. PolitiFact did not acknowledge any problem or offer any response to the email sent to four of its staffers.

IFCN director Alexios Mantzarlis likewise offered no reply. The faulty fact check warranted no mention in the Week in Fact-Checking series Mantzarlis helps author.

If pressed, Mantzarlis would likely respond by saying that the IFCN does not have the resources to check every claim of fact-checking error. Nor does it have the resources to respond to charges its members fail to make needed corrections.

What happens with cases like this one, then, where the fact checker makes a serious error and declines to correct it?

Nothing. The fact checkers carry on as though they did everything right and leave the false information in their fact check and the unfair rating on the “report card” of the politician making the statement.


The Media Research Center and Real Clear Politics recently started projects aimed at holding fact checkers accountable. Our project would strongly augment and improve on those projects, partly through a commitment to establish a bipartisan/multipartisan research team. The new project will prove its trustworthiness with unsurpassed transparency. It will not try to destroy mainstream media fact-checking but instead encourage much higher standards.


Those interested in helping build a better watchdog please visit this page.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.