Lauren Carroll, PolitiFact botch the “Trump Effect”

LaurenCarroll on MSNBC“They asked about 2,000 K-12 teachers who reported in large part that they were seeing an increase in bullying, especially among immigrants, children of immigrants and Muslim students.”

—PolitiFact Senior Writer Lauren Carroll, during an Aug. 27, 2016 appearance on MSNBC Live.

 

Overview

Fact-checker Lauren Carroll passed along false information to the MSNBC audience, reproducing and magnifying errors from her Aug. 26, 2016 fact check published by PolitiFact.

Background

PolitiFact Senior Writer Lauren Carroll touched on three fact-checking issues during her July 27, 2016 appearance on MSNBC Live. One of them involved a fact check of a claim by Democratic presidential candidate Hillary Clinton, who relied on a survey report from the Southern Poverty Law Center to claim “bullying and harassment are on the rise in our schools.”

Carroll’s fact check rated Clinton’s statement “Mostly True,” and Carroll gave the reasoning behind the rating during her appearance on MSNBC Live (transcript ours):

AYMAN MOHYELDIN

So, first of all, does this “Trump Effect” exist? Is this a true statement or false?

LAUREN CARROLL

We rated that claim “Mostly True.” It’s not completely true because the study that was conducted that is the source of Clinton’s claim was not a scientific study. It was more an anecdotal look at what teachers are experiencing across the country.

The study, which was conducted by the Southern Poverty Law Center, found that, er, they asked about 2,000 K-12 teachers who reported in large part that they were seeing an increase in bullying, especially among immigrants, children of immigrants and Muslim students. And the survey did not at all ask about any particular candidate, however they had about 5,000 comments in total and one in five mentioned Donald Trump, which was far more than any of the other candidates were mentioned.

So even though it might not be accurate to say that bullying is on the rise across the country because of Donald Trump, it is anecdotally happening in certain schools.

 

[Update Oct. 4, 2016:
The host of the YouTube video we embedded has removed the video. Carroll’s appearance on MSNBC may be viewed at the Daily Motion website.]

The survey report Carroll mentions was titled “The Trump Effect: the impact of the presidential campaign on our nation’s schools.” It is credited to one author, Maureen B. Costello.

Zebra Fact Check has already completed a detailed look at the SPLC report, finding that it was not designed to detect increases in bullying or harassment. The report  highlights four central findings of its survey:

Here are the highlights:
•More than two-thirds of the teachers reported that students—mainly immigrants, children of immigrants and Muslims—have expressed concerns or fears about what might happen to them or their families after the election.
•More than half have seen an increase in uncivil political discourse.
•More than one-third have observed an increase in anti-Muslim or anti-immigrant sentiment.
•More than 40 percent are hesitant to teach about the election.

 

Note the highlights mention nothing about bullying or harassment. The nature of the survey explains that absence.

The survey asked respondents four agree/disagree questions. None of them asked about bullying or harassment. It followed those with five open-ended questions, one of which asked about bullying: “If you have witnessed bullying or biased language at your school—from adults or students—that mimics the rhetoric of the campaign, please tell us about it.”

The answers to the open-ended questions were published separately from the main report.

We will examine whether Lauren Carroll’s statements on MSNBC find support in the SPLC survey report.

Analyzing the Rhetoric

The MSNBC Live host, Ayman Mohyeldin, asks Carroll if the “Trump Effect” exists.

Carroll answers by telling the audience PolitiFact rated the claim “Mostly True.” As Carroll responds to the question, MSNBC’s stationary chyron reads “Clinton: ‘Bullying and harassment are on the rise in our schools.'” Carroll and MSNBC together send the message that a “Trump Effect” is causing increased bullying and harassment in our schools.

Does the SPLC report show a “Trump Effect”?

The SPLC survey report does not demonstrate any “Trump Effect.” Not even if the SPLC survey adopted scientific methods could it make a reasonable case Trump caused the effects it measures. The SPLC report admits it is not a scientific study, and any attempt it makes to blame Trump for changes in schools is rhetorical, not scientific.

Even worse for PolitiFact and Carroll, the survey offers no real basis for thinking bullying or harassment are rising in schools.

In our earlier evaluation of the SPLC report, we noted that each of the nearly 2,000 respondents had the opportunity to answer question No. 2, asking whether they observed bullying or biased language. Since then, we have looked more closely at the answers to question No. 2.

Of the 849 answers to question No. 2, we counted 23 that one might construe as stating an increase in bullying. Here are two of the strongest examples:

The bullying and fighting (which has increased) is a real disrespect for how students interact.

 

Yes, there has been a huge increase in anti-Muslim speech and bullying of Muslim students. This is of great concern to me and other educators.

 

Some of the 23 examples lack the strength of the two above:

Students have become very hostile to opposing points of view, regardless of the topic. Any division seemingly now elicits anger and personal attacks.

 

By lowering the bar still further, including comments such as “Bullying and misconceptions about certain groups of people is prevalent,” we stretched that number to 55.

Fewer than 1,000 (849) respondents offered answers to question No. 2. It follows that the  supposed increase in bullying and harassment finds its basis in the comments of less than 6 percent of those 849 answers. In terms of the total sample, that represents less than 3 percent.

A representative of PolitiFact appeared on MSNBC and, based on the comments of less than 3 percent of a sample of nearly 2,000 teachers, claimed the teachers reported “in large part that they were seeing an increase in bullying, especially among immigrants, children of immigrants and Muslim students.”

We call less than 3 percent a small part. The SPLC survey does not reasonably support Carroll’s statement.

 

“They had about 5,000 comments in total and one in five mentioned Donald Trump”

The SPLC survey report made a point of emphasizing that about 20 percent of the nearly 5,000 responses to open-ended questions mentioned Trump:

The comments are particularly revealing. The survey did not identify any candidates. But out of 5,000 total comments, more than 1,000 mentioned Donald Trump. In contrast, a total of fewer than 200 contained the names Ted Cruz, Bernie Sanders or Hillary Clinton.

 

Carroll likewise emphasized the frequency with which Trump’s name was mentioned by survey respondents.

Is the statistic significant in the context of an increase in bullying or harassment?

Using a data evaluation tool at Public Radio International, we found a total of 4,796 comments. Of those, 849 comments mention Trump. That’s about one in 5.65. So the true number is closer to one in six than to one in five. We believe the SPLC made its calculation using the total number of Trump mentions instead of the number of comments mentioning Trump.

That error aside, the number of Trump mentions means nothing in the context of this survey. The comments do not focus on bullying or harassment. Comments such as “I listen…I don’t seek to indicate that I find Trump repulsive and dangerous” carry no relevance in measuring either an increase in bullying or any causal role for the Trump campaign.

Carroll’s use of the “one in five” statistic, in the context of asserting an anecdotal increase in bullying or harassment, produces a false impression. It falsely suggests that the number of times Trump was mentioned helps show a “Trump Effect.” With the “Trump Effect” defined in the segment as an increase in bullying or harassment, Carroll ends up misleading the audience. Most of the Trump mentions do not provide any of the type of evidence Carroll implies.

“So even though it might not be accurate to say that bullying is on the rise across the country because of Donald Trump, it is anecdotally happening in certain schools.”

Carroll presents no evidence to reasonably support the idea that bullying is on the rise. She likewise presents no reasonable evidence that Donald Trump serves as a cause of any supposed rise in bullying. We infer that Clinton earned her “Half True” from Politifact by only implying, not asserting, a rise in bullying. The key argument behind Clinton’s argument comes directly from the open-ended answers to the SPLC survey questions.

Note that we do not think the SPLC survey report can support Clinton’s underlying point, that bullying across the country is on the rise thanks to Trump. But let us consider a literal interpretation of Clinton’s remarks. Carroll, though she never explicitly makes the argument, appears to argue that Clinton is really just saying some teachers said bullying is increasing because of Trump. Since teachers say that, the reasoning would go, Clinton’s claim is essentially true.

But does the SPLC survey report clear that low hurdle? Does the SPLC survey report even show that teachers are saying bullying is on the rise in certain parts of certain schools?

Given that only about 20 responses to the question about bullying even asserted a rise in bullying, we do not see how Clinton’s claim can receive a “Mostly True” rating even ignoring Clinton’s false underlying point. Is it irrelevant that less than 3 percent of teachers surveyed reported a rise in bullying? Is it irrelevant that even smaller percentages reported rising bullying targeting minority populations?

Add the idea that Trump caused the bullying increase and the percentage of teachers reporting it drops even further. Indeed, we found not a single clear example matching Clinton’s claim in the survey answers, focusing on the answers most likely to show evidence of increased bullying. We created a page showing our list of the best candidates, arranged roughly in order from strongest to weakest.

Given the actual nature of the anecdotal evidence, PolitiFact could have justified giving the most charitable reading of Clinton’s claim, the literal one, a “Mostly False” rating:

Mostly False – The statement contains some element of truth but ignores critical facts that would give a different impression.

 

We think the small number of responses even vaguely supporting Clinton’s claim counts as a critical fact that would give a different impression. We think the lack of evidence for even one anecdote that unequivocally supports Clinton’s statement also reasonably counts as a critical fact that would give a different impression.

We reached out to Carroll by email asking about her familiarity with the answers to the survey questions, along with what evidence she felt justified her statements on MSNBC. If we receive a response we will update our correspondence page and this article.

Summary

Lauren Carroll’s brief treatment of the “Trump Effect” on MSNBC Live was marked by false statements and fallacious reasoning.

“The study … asked about 2,000 K-12 teachers who reported in large part that they were seeing an increase in bullying”

About 23 teachers out of nearly 2,000 surveyed made statements plausibly construed as seeing an increase in bullying. We suspect Carroll was unfamiliar with the evidence when she made her statement.

False statement icon

“… an increase in bullying, especially among immigrants, children of immigrants and Muslim students”

Fewer than 23 out of 2,000 teachers made statements about an increase in bullying targeting any of the groups Carroll mentions.

False statement icon

“And the survey did not at all ask about any particular candidate”

While it is true the survey did not explicitly ask about any particular candidate, survey respondents were amply cued that the survey was about the effects of campaign rhetoric on Muslims and immigrants. One would be hard-pressed to avoid news of Trump campaign rhetoric about Muslims and immigrants. Carroll omitted mention of evidence the survey was implicitly about Trump’s effect on the campaign.

True Statement icon Missing Context

“They had about 5,000 comments in total and one in five mentioned Donald Trump”

Carroll relied on the SPLC report for this information, which was wrong but not wildly off. It’s true respondents gave nearly 5,000 comments, but the report exaggerated the number of comments mentioning Trump. The proportion of comments mentioning Trump was closer to one in six than to one in five. Regardless of that error, Carroll appears to use the statistic to suggest many of the 1,000 comments identified Trump as a cause of increased bullying or harassment. As such, her argument strikes us as akin to the fallacy of composition.

The great majority of the Trump mentions had no bearing on Clinton’s claims of a “Trump Effect.”

True Statement icon FallacyIcon composition

“So even though it might not be accurate to say that bullying is on the rise across the country because of Donald Trump,”

Carroll’s ambiguous phrasing permits the listener the belief that bullying is increasing, but not necessarily because of Trump. It is trivially true that it might not be accurate to say bullying is on the rise. After all, no solid evidence supports the idea. The evidence Trump caused any change in bullying amounts to even less. Carroll’s lack of clarity would tend to mislead the audience through a fallacy of ambiguity.

True Statement icon FallacyIcon Ambiguity

“It is anecdotally happening in certain schools.”

Finding something “happening in certain schools” serves as one of the very poorest foundations for a warning about a national “Trump Effect” that threatens America’s children. And this claim is a layered nothing, as the anecdotes that clearly support the claim are either nonexistent or nearly so.

We will not judge this claim true or false, recognizing that Carroll’s view of what is “anecdotally happening” may rest on some type of nuance. We expect that any such nuance will disqualify the anecdotes from supporting Clinton’s point about a Trump-inspired trend of growing bullying and harassment in schools.

We charge Carroll with a fallacy of ambiguity pending the presentation of a clear anecdote supporting her claim. If no such anecdote exists then her claim is false.

FallacyIcon Ambiguity

 

 

Reference Material

Carroll, Lauren. “Lauren Carroll Politifact Writer Debunks Lies About Hillary.” Interview by Ayman Mohyeldin. YouTube. YouTube, 27 Aug. 2016. Web. 05 Sept. 2016.

White, Bryan W. “PolitiFact’s ‘Trump Effect’ Failures.” Zebrafactcheck.com. Zebra Fact Check, 31 Aug. 2016. Web. 05 Sept. 2016.

Carroll, Lauren. “Clinton’s Claim about the ‘Trump Effect’ and Bullying.” Politifact.com. Tampa Bay Times, 26 Aug. 2016. Web. 05 Sept. 2016.

Costello, Maureen B. “The Trump Effect: The Impact of the Presidential Campaign on Our Nation’s Schools.” SPLCenter.org. Southern Poverty Law Center, 13 Apr. 2016. Web. 05 Sept. 2016.

Costello, Maureen B. “The Trump Effect: The Impact of the Presidential Campaign on Our Nation’s Schools: Survey Comments.” SPLCenter.org. Southern Poverty Law Center, 13 Apr. 2016. Web. 05 Sept. 2016.

Espelage, Dorothy. “Email Interview with Dorothy Espelage.” Interview by Bryan W. White. Zebrafactcheck.com. Zebra Fact Check, 30 Aug. 2016. Web. 06 Sept. 2016.

Urbanski, Jan. “Email Interview with Jan Urbanski.” Interview by Bryan W. White. Zebrafactcheck.com. Zebra Fact Check, 29 Aug. 2016. Web. 06 Sept. 2016.

Explore the “Trump Effect”” Public Radio International. Public Radio International, 20 Apr. 2016. Web. 06 Sept. 2016.

White, Bryan W. “Email Outreach to PolitiFact’s Lauren Carroll, Sept. 4, 2016.” Zebrafactcheck.com. Zebra Fact Check, 05 Sept. 2016. Web. 06 Sept. 2016.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.