In late 2016 the International Fact-Checking Network issued a statement of principles held in common by its members.*
Today we looked at the IFCN’s methods for evaluating compliance with its code of principles. Our examination of the process, as it is described, left us with concerns. In particular, the methods the IFCN chose to help ensure compliance with its statement of principles lack any strong statement ensuring the transparency of the process.
Note how the middle column of the document describes the evaluation process (bold emphasis in the original):
Evaluation of the applicant’s respect of the code of principles is conducted by external assessors. They are selected for their expertise in journalism and disciplines related to fact-checking in the region where the aspiring signatory operates. They are paid US $350 for each assessment.
The assessors evaluation following this checklist and recommend approval or rejection. The IFCN interim Board* confirms the findings with at least four favorable votes.
Fact-checkers seem tireless in their calls for complete transparency, though their actual efforts at transparency hint that they find themselves too exhausted to comply.
Will the work of the external assessors be made public? That seems a necessary and desirable aspect of transparency, but the IFCN document offers no suggestion that the independent assessors will be named and that their work will be made public.
The IFCN would presumably hand-pick the independent assessors according to the criteria it lists. Of course, if the names are not made public then anybody outside the IFCN would have a tough time judging whether the independent assessors meet the criteria or even truly qualify as independent.
Money will apparently change hands, with independent assessors receiving $350 per assessment. Does the IFCN have a vested interest in making itself look good? Will the work of the independent assessors receive its own independent assessment? Will the payments to the independent assessors predispose them to tell the IFCN what they think it wants to hear?
Zebra Fact Check judges the IFCN’s process unsatisfactory if assessors’ names and their reports are not publicized. Moreover, the Board’s power to confirm the assessment with a supermajority vote from four of its seven member makes a mockery of the concept of fact-checking. That is, unless an unstated rule exists that members of the board will not vote on the accuracy of the report without replicating the work of the independent assessor. A stipulation like that should occur in written form, not as an unstated principle.
Zebra Fact Check’s recommendations:
- The names of independent assessors will appear on their reports
- Assessors’ reports will be made public
- The vote of the Board regarding the findings of the reports will be made public—it is the wrong venue for a secret ballot
- If the Board has the power to “confirm” the findings of the report with its vote, only Board members who have fact checked the findings will receive the privilege of voting
- As an alternative to No. 4, the Board shall retain only the power to support or reject the recommendation of the report, not the power to judge the facts without verifying the facts
- The IFCN shall establish some formal means of transparent public accountability for the independent assessments in addition to those contained in points No. 1 and No. 2
If the IFCN framework fails to include the transparency we fear it lacks, then the IFCN risks making itself ineffectual in accurately judging how well fact-checking entities comply with its statement of principles. Its assessments may end up just as subjective as “Truth-O-Meter” judgments.
Update, Feb. 28, 2017
We tweeted our concerns at the IFCN Twitter account and received a reply from Poynter’s fact-checking director, Alexios Mantzarlis. Mantzarlis affirmed, via Twitter, that the assessors and their assessments will be made public.
Hi there! As signatories get verified, both their applications & their assessment will be published. https://t.co/MZzLrE6fys
— Alexios (@Mantzarlis) February 28, 2017
We applaud that policy.