To & From Michael Wagner (University of Wisconsin-Madison)

Email message sent to Michael Wagner on April 26, 2018:

Following the announcement that the International Fact-Checking Network would serve as a type of quality control organization for fact checkers, I have carried the concern that the verification process lacks teeth. The issue of error correction served as the issue of greatest concern. The IFCN verification process appeared to place emphasis on the existence of a written policy. But how could the IFCN measure compliance with that policy?

Last year I suggested to IFCN directior Alexios Mantzarlis the idea of looking for errors in fact checks and assigning the duty of complaining to a “mystery shopper” of sorts–one that would not tip off to the fact-checking organization that the IFCN was ultimately behind the inquiry. The response to that type of inquiry would provide a potentially good measure of an org’s adherence to its corrections policy.

In 2018, Mantzarlis said the IFCN instead would itself collect complaints about compliance failures and give those to the independent assessors for consideration.

That’s better than nothing, I thought, though potentially far less transparent than I would like.

To test the system I sent a couple of emails complaining about accuracy problems to the IFCN. One concerned an org’s inconsistent standard on describing budget cuts from a future baseline, including an org’s description of one such cut as a cut from current spending levels. Another concerned a fact check that would create in readers the impression that the CHIP program delivers health care in a more cost-effective way than other means. In fact the dollar savings justifying that claim came from increased tax revenues and not from program efficiency.

In both of these cases the fact checks in question committed the types of errors politicians often commit, and the ones we hope fact checkers will catch and bring to light.

With this background you can perhaps imagine my surprise when I read the following from the independent assessment of the organization in question:

Note: Two complaints to IFCN about PolitiFact – from the same source –  were brought to my attention. In both cases, the complaints rest upon disagreements about the editorial judgment PolitiFact made in rendering Truth-O-Meter ratings.

I would say it was incorrect to describe the complaints as “disagreements about the editorial judgment PolitiFact made in rendering Truth-O-Meter ratings.” And I was disappointed to see the description taking place around the principles of fairness and not the principles of open and honest corrections. The best corrections policy in the world does no good at all if an organization resists helpful correction.

I’d like for you to consider answering a question for me on-the-record, Dr. Wagner: What practical steps can the IFCN take to help improve its ability to help ensure fact-checking organizations comply with their own corrections policies?

Reply email from Michael Wagner, received April 26, 2018:

Regarding your question, my assessment of PolitiFact provides the beginnings of an answer in the open and honest corrections section where I wrote, “given the prominence and quality of PolitiFact, it might be worth considering including a page that publishes complaints and questions about the work PolitiFact does.” This statement operates under the assumption that corrections will likely stem from complaints and/or questions about PolitiFact’s work on any given fact-check or series of fact-checks. More specifically to your questions, then, IFCN might consider suggesting or requiring fact-checking organizations publish some or all of the complaints and questions they receive about their stories.

Regarding the messages to PolitiFact of which I was made aware, the subject line on one of the complaints IFCN shared with me was “Trump-O-Meter entries mislead on Medicaid cuts” and another noted a “big error” in a fact-check but ended the message by seeking a “correction” or “response.” Since one complaint specifically highlighted a rating and the other sought relief about a correction or response that would be likely to change the rating, I elected to note questions about PolitiFact’s work in both the nonpartisanship and fairness section and in the open and honest corrections section (where my suggestion was offered to PolitiFact to publish the questions and complaints they receive).