“Everyone is entitled to his own opinion,” said the late Sen. Patrick Moynihan, “but not to his own facts.”
Fact checkers like PolitiFact have a way around Moynihan’s maxim. They first choose which claim to check, then choose how to check the claim.
Both steps feature subjective judgments. The end result? The fact checkers choose their own facts. PolitiFact’s Feb. 3, 2020 fact check of Rep. Val Demings (D-Fla.) serves to illustrate.
Demings served as one of the House Managers during President Donald Trump’s impeachment trial. PolitiFact fact checked Demings as part of its coverage of that trial.
PolitiFact chose to look at Demings’ claim that Ukraine has not yet received all the U.S. aid it was supposed to receive after President Trump froze the aid in July 2019. In its fact check PolitiFact did not offer a source for Demings’ remarks and offered no account of the surrounding context.
Demings Before the Senate
Zebra Fact Check located the relevant material via C-SPAN and clipped a video to present Demings’ comments in full context.
Demings took the floor to address a question from Senate Democrat Patrick Leahy. Leahy wanted to hear the House address claims from Trump’s defense team that freezing the aid had no ill effects for Ukraine and the Trump administration had treated Ukraine more favorably than did the Obama administration.
Demings argued the United States owed a defense commitment to Ukraine, that 15,000 Ukrainians had died in the conflict with Russia and that Ukraine still had not received all the promised aid.
Demings did not address the second part of Leahy’s question.
Hocus Pocus, Magic Focus
Subjectivity Phase I
Upon viewing the full context of Demings’ speech, we quickly realized she made a claim PolitiFact had, in the past, given a “False” rating.
During the 2016 presidential campaign, Republican presidential candidate Ben Carson said the United States promised to defend Ukraine in return for its divestment of its nuclear arsenal. PolitiFact called Carson’s claim false:
Carson said Ukraine was a nuclear-armed state and that it gave up those arms with the understanding that the United States would protect it. While it is true that about 1,900 nuclear warheads were on Ukrainian soil, experts make it clear that Ukraine merely possessed the weapons and had no ability to use them.
Meanwhile, the security agreements struck after the fall of the USSR specifically avoided committing the United States to protecting Ukraine, instead committing not to attack the newly created independent nation.
We rate this claim False.
Compare Carson’s claim with what Demings offered before the Senate (transcription ours):
We promised Ukraine in 2014 that if they gave up their nuclear arsenal that we would be there for them, that we would defend them, that we would fight along beside them.
We find it remarkable that PolitiFact would pass over Demings’ repetition of a claim it found “False” in the past. Fact checkers look for falsehoods to expose. How could PolitiFact overlook a falsehood it had ruled on just a few years ago?
Demings’ reply to Leahy included mention of 15,000 Ukrainian deaths. But what was the relevance of that number to her argument if the United States carried no obligation to defend Ukraine?
Were a substantial number of those deaths associated with Trump’s 2019 freeze on aid to Ukraine?
The United Nations put the Ukrainian death toll at nearly 10,000 back in December 2016, before Mr. Trump was sworn into office.
Had Demings included that information in her speech, it would have answered Leahy’s question in a way that seems favorable to Mr. Trump. The bulk of the deaths occurred between 2014 and 2017 with President Obama in the White House. And the Obama administration never approved aid to Ukraine that included lethal weapons.
Demings’ reference to 15,000 deaths served to mislead, given that Trump’s policies had nothing to do with about 10,000 of them.
Again, PolitiFact showed no interest in exposing her deception.
At Last, Something Worth Fact-Checking?
After passing over two perfectly legitimate fact check topics, PolitiFact settled on whether some aid to Ukraine had yet to reach Ukraine.
In the context of Demings’ presentation before the Senate, the still-undelivered aid served to illustrate the harm Trump did to Ukrainians.
PolitiFact’s fact check concluded that Ukraine has yet to receive about 7 percent of the military aid Congress designated for Ukraine. Though Demings stopped short of making the connection explicit, she apparently argued that the 7 percent shortfall represented an adverse impact on Ukraine.
How does that stack up against Trump failing to live up to the supposed U.S. commitment to defend Ukraine or the 15,000 Ukrainian deaths?
For PolitiFact, it was the top priority in Demings’ speech.
Subjectivity Phase II
After subjectively deciding what claim to check, the fact checkers subjectively evaluate the claim. The subjective evaluation tends to involve facts, but fact checkers tend to treat the facts subjectively. That’s especially true of fact checkers like PolitiFact that use subjective rating scales like the “Truth-O-Meter” to express their rulings.
We’ll look at a number of problems with the Demings fact check to illustrate that point.
The Main Problem
Sometimes PolitiFact considers the context of a claim. Sometimes it apparently doesn’t. But PolitiFact’s founding editor, Bill Adair, said “To assess the truth for a numbers claim, the biggest factor is the underlying message.” So, according to Adair, fact checkers should consider the context of a claim to assess the point of the statement.
In his 2011 article, Adair gave an example where a politician (President Obama) was off on the numbers but received a “Mostly True” rating thanks to his underlying message.
Demings offered no plausible illustration of harm to the Ukrainian people other than the bare deficit in dollars. PolitiFact, in effect, completely ignored her underlying point.
In fact, PolitiFact offers no specific rationale at all for the “Mostly True” rating. Why not simply “True,” if PolitiFact found no identifiable fault with Demings’ claim?
The fact check literally contains nothing to help the reader specifically understand the rating. PolitiFact’s summary paragraphs help illustrate:
Demings said “all of the aid has not arrived” to Ukraine.
The Defense Department told us that of the $250 million, about $17 million has not yet been obligated. That’s about 7%. When Trump unfroze the aid in September, that left little time until the end of the fiscal year that month. Congress passed a continuing appropriations bill so the funds could be carried over into 2020.
We rate this claim Mostly True.
There’s no sign of any consideration of any underlying message, so it’s anybody’s guess why Demings’ claim failed to receive a “True” rating. PolitiFact’s description appear to equate to “It was true, so we rate it “Mostly True.”
Various Additional Problems
Incomplete Source List
As noted above, PolitiFact neglected to provide a specific citation leading to the claim it was fact-checking. The fact check also contains internal evidence of interview material that failed to reach PolitiFact’s source list.
PolitiFact offers apparently fresh reporting from a Pentagon spokeswoman, Lt. Carla Gleason, mentioning in the body and conclusion of the fact check. Yet the citation list mentions no interview with a Pentagon source.
Misleading Story Structure
PolitiFact structured its story in a way that makes the Trump administration’s failure to obligate aid funds appear illegal:
About 93% of the $250 million designated for Ukraine by Congress has been obligated, said Pentagon spokeswoman Lt. Col. Carla Gleason. That means about $17 million has not yet been obligated.
Officials at the Office of Management and Budget have argued that it’s common for agencies to not fully spend their money by the end of the fiscal year, including for Ukraine aid. But the Government Accountability Office concluded that the office violated the federal Impoundment Control Act by withholding security assistance to Ukraine.
The first paragraph above continues the fact check’s focus on aid funds not yet obligated.
The second paragraph offers the appearance of a smooth segue. But the OMB argument pertained to the Trump administration freeze on aid, not in particular to the small amount the Department of Defense has not yet obligated. But it’s worth noting the argument could apply to the $17 million PolitiFact identified.
Likewise, the subsequent opinion from the Government Accountability Office has to do with the aid freeze, not to funds not yet obligated. It’s not clear at all how the GAO opinion would apply to leftover Ukrainian aid funds reauthorized by Congress for the present fiscal year.
PolitiFact helped ensure confusion by going back to the topic of funds not obligated in the next paragraph.
The second paragraph offers reasonable background on the aid freeze, but PolitiFact should have placed that background where it would not cause needless confusion.
The Uncommon Denominator
Why did PolitiFact figure the unspent aid as 7 percent? In other words, why did PolitiFact use the Pentagon’s share of the Ukrainian aid package as the denominator for calculating the percentage?
PolitiFact did not explain the calculation as such. Instead, PolitiFact simply described how it did the calculation:
Trump’s hold on the aid last summer sent it off the usual path. The question about what federal dollars have not yet arrived relates to defense money.
As explanations go, PolitiFact’s ranks somewhat below the Monty Python instruction on how to play the flute: “Well, you blow in one end and move your fingers up and down the outside.”
The fact that the delayed federal dollars came from the Pentagon share of aid does not require using the defense dollars as the denominator. The most likely reason for not using the total amount of the aid package as the denominator was probably the fact that it made the unspent percentage look bigger.
Spin, in other words.
As a percentage of the total aid package, the unspent amount accounted for 4.3 percent.
PolitiFact subjectively passed up two Demings claims, ripe for fact-checking, in favor of one that carried questionable relevance.
In its fact check, PolitiFact subjectively downplayed the context of Deming’s remarks and produced a ruling (“Mostly True”) in keeping with a subjective interpretation of PolitiFact’s principles for handling numbers claims. That is, PolitiFact simply ignored the point Demings tried to bring home with her claim about unspent Ukrainian aid.
Mistakes regarding the source list simply count as mistakes. But PolitiFact’s subjectivity led to the odd and misleading editing decision we described, as well as its choice to express the unspent aid dollars as a percentage of defense aid.
In effect, PolitiFact chose its own facts.
Correction Feb. 7, 2020: In our section describing “The Main Problem” the third paragraph featured a misspelling of Rep. Deming’s last name using a double “m.” We have fixed that problem.