Though I haven’t especially pushed it in the few promotional statements put forth about Zebra Fact Check, it was never lost on me that the name lends itself to fact checking other fact checkers. “Zebra,” of course, often serves as a mildly derisive nickname for sports referees who run about, at least in American football, in uniforms that feature black and white stripes.
Today I plan to publish my first specific fact check of a fact-checking entity, PolitiFact Florida. The story is done. I’m just giving PolitiFact staffers some time to offer comments that might help illuminate the process PolitiFact used, or perhaps even justify its behavior.
In this blog post I’ll illustrate part of the importance of fact checking other fact checkers.
Why?
On Jan. 4, PolitiFact Florida published a fact check of a tweet by Sen. Marco Rubio (R-Fla.). A “tweet,” for the uninitiated, is a message posted on Twitter, a messaging service that normally limits commentary to 140 characters.
A fact checker ought to have good reason for fact checking comments made on Twitter. The format by its nature limits the degree to which a user can supply context.
PolitiFact offered enough background for its story so that we can sketch PolitiFact Florida’s reason for rating Rubio’s tweet:
On the afternoon of Sunday, Dec. 30, (Rubio) tweeted: “Report that #GOP insisting on changes to Social Security as part of #fiscalcliff false. BTW those changes are supported by @barackobama.”
The next day, liberal columnist Paul Krugman said Rubio lied about the negotiations, and that “numerous reports tell us that McConnell did in fact make precisely that demand,” referring to the Senate Republican Leader Mitch McConnell of Kentucky.
New York Times opinion columnist Krugman is probably the reason PolitiFact Florida took notice of the story. Krugman asserts that Rubio lied. Journalists doing fact checking will tend to notice that type of thing even if they didn’t pay much attention to Rubio’s tweet.
But at this stage of the game PolitiFact Florida has a choice to make. Should it rate Krugman’s claim or Rubio’s? Or both?
Selection bias
This is where selection bias comes into play. A claim of somebody lying is probably sexier than a tweet about legislative negotiation strategies. But is it fair to rate a claim that somebody lied? Doesn’t that sort of claim more qualify as opinion? Krugman’s an opinion writer, after all. Why rate one of his statements? Rubio’s the politician, right?
I’d say that to a neutral observer, Krugman’s claim should look like it lacked a firm foundation. He equivocated between Rubio’s term “insisting” and the more ambiguous term “demand.” He used popular news reports as his evidence, invoking a form of the bandwagon fallacy. But could PolitiFact justify giving Krugman a “True” or “Mostly True” rating for his claim if Rubio’s statement is “Half True”? Even giving Krugman a “Half True” would dent his comparatively solid history of PolitiFactualness.
PolitiFact’s solution to the selection question tracks what we might predict from a liberally biased organization. Find a way to rate Rubio “Half True” or lower even if his statement is sufficiently justified to wear a “True” rating. Don’t rate Krugman at all. Instead, uncritically paraphrase Krugman (more than once, if possible) calling Rubio a liar. Giving Rubio a “Half True” or lower will implicitly support Krugman’s assessment despite its flimsy foundation.
Did staffers at PolitiFact use that specific reasoning? I don’t know. But it doesn’t matter. The end effect is the same either way.
