Fake accountability and the International Fact-Checking Network

The International Fact-Checking Network has yet to figure out how to balance its role as a fact-checking advocate with its role in fact-checker accountability. The IFCN peddles the image that it verifies compliance with its Code of Principles. In practice, its accountability measures count as incredibly weak. The IFCN’s 2021 treatment of its Poynter Institute stable mate PolitiFact exemplifies the problem of fake accountability.

On Fairness and Corrections

Of the criteria the IFCN claims to police with its verification system, two stand out as the most important. First, the IFCN claims to verify fact-checking organizations act in a spirit of non-partisanship and fairness. Second, the IFCN claims to verify fact-checking organizations “scrupulously” follow an open and honest corrections policy.

After the IFCN created a system inviting public complaints about the conduct of its verified signatories, Zebra Fact Check looked to test that system. We focused on the two criteria above, especially corrections policies, because of the ease of proving those problems.

Proof Required by the IFCN

When a fact-checking organization seeks verified status with the IFCN, the IFCN asks for evidence of non-partisanship and fairness. The IFCN tells applicants to pick 10 fact checks showing their non-partisanship. Applicants must also offer a short explanation:

To show they meet these criteria, applicants should:

Provide in your application a selection of 10 fact checks that you believe demonstrate your non-partisanship, and an explanation of how these show that you use the same high standards of evidence for equivalent claims, do not unduly concentrate your fact-checking on any one side, consider the reach and importance of the claims you select to check, follow the same essential process for every fact-check and let the evidence dictate your conclusions.

Applicants, then, may show the IFCN a pattern of non-partisanship and fairness by hand-picking 10 fact checks as examples.

The IFCN also asks for proof that fact-checking organizations scrupulously follow their policies on corrections:

Provide a short statement in the application about how the policy was adhered to over the previous year (or six months if this is the first application) including evidence of two examples of the responses provided by the applicant to a correction request over the previous year.

Applicants, then, may show the IFCN a pattern of compliance with their policy on corrections using two examples.

Such low standards from the start concerned us with their potential for fake accountability. The keys to journalistic integrity, objective fact-finding and zealous corrections, escape serious scrutiny.

On the bright side, the ease of demonstrating compliance led us to think we might prove non-compliance just as easily.

PolitiFact’s Correction Policy Failures

As soon as Zebra Fact Check learned the IFCN would welcome public complaints about its fact-checking organizations, we resolved to test its system. We focused on PolitiFact to take advantage of the appearance of a conflict of interest. As we noted at the outset, the Poynter Institute owns PolitiFact and the IFCN. It looks bad if the IFCN appears to take it easy on PolitiFact.

As we were aware of PolitiFact’s history in struggling with corrections, we focused on correction policy failures. We looked to identify a mistake, inform PolitiFact of the mistake and then look for a correction.

If no correction resulted after a reasonable time, typically one week, we lodged a formal complaint to the IFCN. We sent the IFCN complaints about PolitiFact in 2018 and 2019. The IFCN handled complaints poorly both years, with cursory mention in the 2018 report and total omission from the 2019 report. Find much more on that story here.

PolitiFact Complaints, Year Three

Near the end of 2019, the IFCN announced revisions to its Code of Principles and associated policies on verification standards. We took that as a hopeful sign. Perhaps the IFCN’s calls for “scrupulous” observance of a corrections policy were serious.

We made sure every complaint we lodged named the part of the code PolitiFact breached. Below, we list each documented complaint alleging a failure on corrections.

1. Botched a story on Social Security and the federal deficit
2. Published two fact checks with different totals for President Obama’s inherited budget deficit.
3. Contradicted itself by saying real wages increased but were outpaced by inflation.
4. Contradicted itself in an article about the Michael Flynn case.
5. Zebra Fact Check documented PolitiFact’s failure to maintain its list of corrected stories.
6. Issued a ruling that violated PolitiFact’s statement of principles.
7. Heavily modified a key paragraph and skirted its corrections policy.
8. Zebra Fact Check sent notice of two more examples of PolitiFact skirting its corrections policy.
9. Issued a false report about President Trump’s statement regarding “very fine people.”
10. Botched two explainer articles on intraracial murder by overlooking/burying a key statistic.
11. Miscounted total U.S. deaths during WWII, substituting the number of armed service deaths.
12. PolitiFact’s video version of a fact check used an improper paraphrase.

Zebra Fact Check sent no fewer than 12 formal complaints about corrections to the IFCN since PolitiFact’s 2019 evaluation. We do not know how many were excluded for any reason, thanks to the lack of transparency at the IFCN.

For most of our 12 examples, PolitiFact made no correction at all.

What Level Proof for Non-compliance?

The IFCN’s Code of Principles calls for “scrupulous” adherence to an open and honest policy on corrections. But the IFCN’s 2021 report on PolitiFact appeared to move the goalposts:

Additional explanation from IFCN Staff

The complaints that this applicant has received are not substantiated in showing a petter [sic] on violating any criteria, therefore do not constitute a violation of the Code of Principles. 

The IFCN Code calls for “scrupulous” adherence to an open and honest corrections policy. But its one-line dismissal of the complaints seems to say a complaint must show a pattern of transgression. If the idea was to change the requirement of the Code, the change has yet to find its way to the text of the Code of Principles:

Signatories publish their corrections policy and follow it scrupulously. They correct clearly and transparently in line with the corrections policy, seeking so far as possible to ensure that readers see the corrected version.

The Code goes on to stipulate:

6.3 Where credible evidence is provided that the applicant has made a mistake worthy of correction, the applicant makes a correction openly and transparently, seeking as far as possible to ensure that users of the original see the correction and the corrected version.

The IFCN seems to say violations of the Code do not count as violations of the Code unless the violations occur in a pattern. It ends up looking like the IFCN re-interpreted the Code to erase PolitiFact’s violations.

How many examples does the IFCN require to show a pattern of violation? Apparently that’s for the IFCN to know and for complainants to find out through trial and error.

Fake Accountability

Merriam-Webster says “scrupulous” means “having moral integrity : acting in strict regard for what is considered right or proper.” We don’t see how a corrections policy that fails to correct errors can count as scrupulous, regardless of whether such exceptions occur in a pattern. Failing to fix any known substantive error over a period of time all by itself forms a pattern of transgression. Every day one known error persists extends the pattern. Journalistic integrity means when the journalist finds out about an error it gets fixed.

It’s fake accountability when the IFCN says it verifies compliance with a Code of Principles, including “scrupulous” adherence to an open and honest corrections policy, but then freely allows exceptions. And the IFCN uses that fake accountability to deceive the public about the reliability of fact checkers.

We’d like to see an end to the IFCN’s fake accountability.


Note: We plan a second part to this story, looking at how the IFCN describes its process for handling complaints and comparing that to the IFCN’s descriptions of what it did in PolitiFact’s case in 2021.

Update Aug. 26, 2021: Touchups to grammar and punctuation
Correction Aug. 27, 2021: Our first correction request was listed as “Botched a gender wage gap story.” But the complaint was actually about a different blunder from the defunct PolitiFact Oregon site, on Social Security and the deficit. New wording reflects that.