After finding multiple problems with a PolitiFact Virginia fact check of Democrat Bobby Scott, Zebra Fact Check sent a message to PolitiFact intended to draw attention to the problems.
After sending the first message we noted PolitiFact Virginia made some changes to the fact check. So we sent a second message to PolitiFact highlighting the lingering concerns.
The messages to PolitiFact follow in .pdf form.
Note: We do not believe we sent this to the International Fact-Checking Network as an example of PolitiFact failing to follow its policy on corrections. But, point of fact, PolitiFact Virginia to date has not followed PolitiFact policy by using the tag “Corrections and Updates” on its fact check. The fact check does have a correction notice appended.
Dear Truth-O-Meter,
The March 13, 2019 fact check by PolitiFact Virginia of a gender pay gap claim needs a near-total revision.
The fact check effectively ignores strong evidence U.S. Rep. Bobby Scott was repeating a common error, representing the raw wage gap for all jobs as a wage gap for doing the same job.
1) Scott’s staff said his information came from Census Bureau data, which is not broken down by job categories. PolitiFact typically uses staff statements as an appropriate standard for interpretation.
2) PolitiFact Virginia ignores the fact that “similar” may be used as a synonym for “same.” Instead of using the overlapping meaning to help interpret Scott’s statement, PolitiFact asserts that “similar” and “same” are so different that it means Scott’s statement was nuanced (PolitiFact Virginia’s term).
3) The supposed nuance in Scott’s statement results in PolitiFact Virginia trying to fact check Scott’s statement using raw gender pay gap numbers for select job categories.
A) PolitiFact Virginia settled on an undisclosed list of 29 job categories. One might assume that the Bureau of Labor Statistics somehow directly guided that selection process.
B) In trying to follow the steps PolitiFact Virginia took for its fact check, I found that the hierarchical structure in the BLS data apparently does not feature any clear list of 29 job categories.
C) I found a five-tier system, with the top tier featuring 5 groups, the second tier 14 groups, the third tier 12 groups and the fourth tier 352 groups.(I did not count groups in the fifth tier). The author has not responded to my request that he identify the 29 categories he used.
PolitiFact Virginia’s exercise in crunching BLS numbers almost certainly has no relevance to Scott’s statement based on Census Bureau numbers, and even if Scott, contrary to what his staff indicated, relied on BLS.gov numbers the methodology PolitiFact Virginia used apparently lacks both rhyme and reason.
Finally, the PolitiFact Virginia fact check relies in principle on a long-running problem with PolitiFact’s treatment of the gender wage gap. The evidence has made clear that the raw gender wage gap has little relation to the part of the gender wage gap caused by gender discrimination. PolitiFact takes a friendly view of politicians who, in the context of “equal pay for equal work” legislation, just happen to mention the raw gender wage gap.
There’s hardly any reason to use the raw gender wage gap in the context of gender discrimination in wages apart from a desire to illicitly magnify the size of the gender wage gap caused solely by discrimination.
There’s a name for this technique. Its ubiquitous use in political speech on the gender wage gap makes it likely a deliberate deceit. PolitiFact routinely grades it “Mostly True.”
A revised version of the fact check should give appropriate attention to the overlapping pools of meaning for the terms “same” and “similar.” It should give more weight to statements from Scott’s staff that he based his comments on Census Bureau data. If it finds any reason to interpret Scott’s statement in terms of BLS data, it should perform that research in a sensible way and explain it in a way that allows the reader to duplicate its results. Summaries of the CONSAD research should not downplay the fact that the researchers do not claim to have taken all factors affecting the raw pay gap into account. The estimate of 5 percent to 7 percent that may occur from gender discrimination should be expected to drop when accounting for any of those additional factors. The introduction to the report, in fact,
says “the raw wage gap should not be used as the basis to justify corrective action. Indeed, there may be nothing to correct. The differences in raw wages may be almost entirely the result of the individual choices being made by both male and female workers.”
It’s worth noting that the PolitiFact Virginia fact check references the CONSAD report but does not include it on the list of sources in the sidebar. Moreover, the embedded link to the report is dead (I used an Internet Archive version of the same URL to link it, above).
Dear Truth-O-Meter,
Update to previous message
It seems that at some point following my last message to you that PolitiFact Virginia has again tweaked its fact check. Instead of 29 job categories PolitiFact Virginia has trimmed the number to 22. This agrees with my research showing BLS uses a system that places all U.S. jobs into 23 categories. The agreement occurs because one of the categories is specific to military jobs and BLS does not publish information on those jobs in its current population survey. That leaves 22.
With the categories identified, it was possible to proceed further with the attempt to re-enact PolitiFact Virginia’s process. Unfortunately, I had no luck matching PolitiFact Virginia’s “Overall in 2018, women earned 78.7 percent as much as white men in the same areas of work.”
The BLS system does allow users to determine the mean figures for white men as well as women of all races. I calculated a raw gap of 77.8 (median figure) percent for all jobs. I could imagine no well-founded approach to the math for calculating the median wage figure by averaging other median figures for broad job categories, but tried one such method which yielded a figure of 79.7 percent for the raw gap between white men and women of all races.
Perhaps PolitiFact Virginia reached a different result by including 29 categories in its original calculation and declining to re-calculate when the number shrank to 22.
In any event, the gap PolitiFact Virginia calculated is highly suspect, and in the best case scenario PolitiFact Virginia is effectively saying that Scott stated the raw wage gap is about 80 cents on the dollar (for women of all races compared to white men) and his use of “similar jobs,” despite the contextual details mentioned in my earlier message, could not be taken to imply that the gap occurs where the women are doing the same work as men.
It’s likely that PolitiFact Virginia used totally inappropriate methods in trying to confirm Scott’s statement. The fact check certainly makes statements that blur the line between mean and median figures, such as this one:
Overall in 2018, women earned 78.7 percent as much as white men in the same areas of work.
Averaging can justify a figure described as an “overall” comparison of earnings. For a median figure such language is misleading.
To summarize, PolitiFact Virginia’s fact check offers readers an incoherent and nonsensical attempt to excuse Mr. Scott’s slightly ambiguous attempt to say women earn just 80 cents on the dollar compared to white men for doing the same work. The term “similar” is not entitled to the descriptor “nuanced” if explained as an attempt to compare earnings in the 22 broad areas of employment described by BLS. Those broad areas of employment tend to contain subcategories of very different kinds of work. The main idea of the fact check is wrong and the would-be supporting details compound the error.
The fact check should have included a spreadsheet (or chart) something like this one to help explain how it calculated its figures.
https://docs.google.com/spreadsheets/d/10T5MbD8VSkwirtf-Was3xtCVVI8esLqLGxuldT97vq0/edit?usp=sharing
Thanks.