Fact checkers take odd approach to Trump Southwest border claim

President Donald J. Trump has said illegal border crossings dipped 78 percent under his administration.

Mainstream fact checkers have lumped that claim in with an associated claim, that immigration has gone up under other administrations. Their fact checks have tended to focus on this second claim instead of Trump’s claim of a 78 percent decrease.

PolitiFact piqued our interest in the subject when it tweeted a graphic implying Trump’s 78 percent claim was flatly false.

PolitiFact’s explanation for the “False” rating deals only with Trump’s claim about a 78 percent decrease. That feature of the graphic leads readers to think that the “False” rating applies to the first part of the claim, that Southwest border crossings have dropped by 78 percent.

PolitiFact often has no problem giving a cherry-picked statistic a rating of “Mostly False” or higher. In this case, PolitiFact and the Washington Post Fact Checker adopted weak approaches in checking the 78 percent claim.

Finding a Baseline

When a fact checker tests a claim of a percentage increase or decrease, finding the baseline stands as the most important step. The fact checker should try to avoid cherry-picked baselines in favor of more representative baselines.

In testing Trump’s claim of a 78 percent drop in illegal Southwest border crossings, we would recommend looking at the same type of crossings for the same part of the year for the Trump administration and the preceding administration.

We would guard against the assumption that the year preceding the Trump administration was representative of the Obama administration.

If we have no numbers for illegal border crossings, we would plan to explain to the reader the reasoning behind using a proxy statistic, such as border apprehensions.

How did PolitiFact and the Washington Post Fact Checker approach the issue?

PolitiFact

PolitiFact produced two items dealing with Trump’s claim. The most recent version occurred in a summary assessment of Trump’s claims at a campaign rally in Phoenix. The summary assessment for the 78 percent claim is brief, so we include the whole thing to ensure that we omit no relevant context:

Border “down 78 and almost 80 percent”

Border apprehensions have gone down under Trump, but not by that much.

When Trump falsely claimed something similar in July, we found that the closest number to 78-80 percent would be based on cherry-picked numbers of the highest number of apprehensions, in November, to the lowest number, in April.

The latest figures from U.S. Customs and Border Protection show that in July there were 18,198 apprehensions at the southwest border, a 46 percent decline from July 2016. While apprehension numbers are low under Trump, they began increasing in May.

 

PolitiFact uses July 2016 as its baseline for a decrease when compared to July 2017. What makes July 2016 representative? PolitiFact offers no explanation. On the contrary, PolitiFact says border apprehensions started increasing in May 2017. Readers might infer that before July the decline in apprehensions was greater than the 46 percent PolitiFact chose to emphasize. But PolitiFact’s presentation may lead readers to infer instead that Trump’s influence on decreased border apprehensions has slipped and will not hold.

PolitiFact’s claim that “Trump falsely claimed something similar in July” links to its August 3, 2017 rating of a Trump claim from July 28, 2017:

Where does the 78 percent come from? We asked the White House but did not get a response.

From June 2016 to June 2017, apprehensions at the southwest border went down 53 percent.

U.S. Customs and Border Protection data also show that from November to June (latest available figures), apprehensions by border patrol agents declined about 66 percent. From January to June, it declined 49 percent. From Trump’s first full month in office, February, to June, it went down 14 percent.

 

PolitiFact uses a number of baseline comparisons. It compares June 2016 to June 2017. It compares November 2016 to June 2017. It compares January 2017 to June 2017. It compares February 2017 to June 2017.
None of the comparisons represents an attempt to find a representative baseline and a legitimate difference from that baseline. It is as though PolitiFact illustrates the weakness of using cherry-picked statistics by showing the variation that results by using different approaches to cherry-picking. While that lesson carries some value, why not model checking the facts in the best way possible and teach an even more valuable lesson?

At the tail end of PolitiFact’s assessment of Trump’s “78 percent” claim, it references the Washington Post Fact Checker’s findings on the same question:

As the Washington Post’s Fact Checker noted, Trump’s claim of apprehensions nearing 80 percent is clearer when comparing cherry-picked figures: the highest point in fiscal year 2017 (November) to the lowest point (April). That yields a 76.4 percent decline.

 

Oddly, the Washington Post Fact Checker’s reasoning ended up, uncredited, in the PolitiFact graphic that grabbed our interest in the first place.

How did the Fact Checker do in checking the facts?

The Washington Post Fact Checker

Before we get to the Fact Checker’s evaluation of Trump’s “78 percent” claim, we note that the Fact Checker did far better than PolitiFact in describing the context of border apprehensions. The Fact Checker embedded an important chart from Customs and Border Protection and explained it clearly and accurately.

The Fact Checker’s explanation for using border apprehensions as a proxy statistic for illegal border crossings was weak, unfortunately. Assuming more vigorous policing of the border by the Trump administration, lower border apprehensions reflect an even greater reduction in illegal border crossings. Fact checkers should point out that illegal crossings may have decreased more rapidly than border apprehensions.

Seventy-eight percent?

The Fact Checker first takes the PolitiFact approach toward the data, using various non-representative methods to measure the decrease in illegal Southwest border crossings. The Fact Checker spells out at the end the same implicit conclusion PolitiFact reached (bold emphasis added):

His figure is consistent with the change in apprehensions data from the highest point in fiscal 2017 (November 2016, at 47,210) to the lowest point (April 2017, at 11,126), which is a 76.4 percent decline. That is a cherry-picked figure that shows the biggest possible decline in a given fiscal year — including the months before he took office. When you compare the data from his first full month in office (February 2017) with the latest data (June 2017), the decline is between 8 and 14 percent.

Another way to look at this data is using year-over-year trends. From June 2016 to June 2017, the number of people apprehended or deemed inadmissible declined 52.6 percent. As FactCheck.org noted, it’s still a significant decline, but not as high as 80 percent.

Clearly, monthly figures are quite susceptible to cherry-picking. So let’s take a broader look at the data.

 

The Fact Checker’s broader look at the data includes no apparent attempt to use the data to reach a reasonable conclusion about the accuracy of Trump’s “78 percent” claim. The Fact Checker, for whatever reason, missed the opportunity.

What if the fact checkers had checked the fact in question?

Using the method we sketched at the outset, Southwest border apprehensions have decreased by about 56 percent under the Trump administration.

We derive that figure by comparing total Southwest border apprehensions/inadmissables under Trump to the average number for the same months for the Obama administration from 2012 to 2016.

In literal terms, Trump’s claim of a 78 percent drop is likely false. The number is closer to 50 percent, reasonably judged as a comparison to the previous administration. But the fact checkers made Trump’s claim appear further from the mark by shirking their responsibility to sensibly check the facts.

Bookmark the permalink.

Leave a reply here or on our Facebook page