MANILA, Philippines – A question often asked of and by fact-checkers is: how do you know if the fact-checking program is working?
Fact-checking group Science Feedback attempted to answer this question by analyzing Facebook data captured via CrowdTangle and BuzzSomo. They presented their findings at During Global Fact 9, a conference of fact-checking organizations held in Oslo, Norway, June 21-25.
Science Feedback is a French fact-checking organization that focuses on science-related information. In a nutshell, the group found that Meta’s enforcement of its policies against repeat offender accounts, defined as pages or groups that posted at least two of its posts as fake within 90 days, results in a reduced engagement on these posts during the lead time. However, they also found that it did not change the behavior of repeat offenders over the long term, according to Emmanuel Vincent, director of Science Feedback.
“For those who do it unintentionally, there are anecdotes (that they) make corrections,” Vincent said during a breakout session at the recent Global Fact 9. But that’s not the case with groups and pages spreading false information on a massive scale. “They do it repeatedly. They really know what they are doing… These pages and groups have no intention of changing their behavior in any way.
Because of the discovery, Vincent recommended that Meta, Facebook’s parent company, consider more information when enforcing rules against perennial disinformation providers. “Ninety days is extremely short. We have actors we know have released over 100 in the past 2 years. This is valuable information.
He also recommended looking at other misinformation signals other than fact-checking. For example, what could be analyzed are the narratives that are pushed in certain groups, he said. “Many groups have gotten good at not saying explicitly that vaccines are going to kill your children. Yet they simply say that someone died and already took a vaccine.
Valuable data source
As part of its research, Science Feedback used several data collection tools. One of them is CrowdTangle, a Meta-owned public information tool that makes it easy to track, analyze, and report on what’s happening with public content on social media.
Acquired by Facebook’s parent company in 2016, CrowdTangle has been used by newsrooms to monitor and benchmark the performance of public assets and content across Facebook, Instagram. When Facebook restricted access to its APIs in 2018 following the Cambridge Analytica scandal, researchers, journalists and fact checkers had to rely on CrowdTangle to monitor the spread of disinformation and misinformation on these platforms. .
“We used CrowdTangle to identify all of the pages, all of the groups that are posting misinformation and looked at stats on posting behavior and whether they are gaining or losing viewership,” Vincent told Rappler.
It’s a matter of transparency, he says. “In our research, we were able to show how much Facebook reduces the visibility of posts in these accounts of repeat offenders. We were able to transparently verify that Facebook implements this policy. Otherwise, there’s no way to know. “We really need this kind of data if we want to be able to measure what’s happening on the platform.”
Faktisk, a fact-checking organization based in Norway, also used data from CrowdTangle to map the spread of false claims. In an article published in 2020, the group published a story showing how a controversial COVID-19 article published by Norway’s Public Broadcasting Network spread misinformation to millions of people. The same article identified so-called “superspreaders” on Facebook and Twitter.
Rappler has also used CrowdTangle data extensively to investigate disinformation peddlers as well as those attacking journalists on Facebook. Some of these investigations have led to dismantling.
Access to CrowdTangle is one of the main benefits of the fact-checking partnership with Meta, Faktisk’s Kristoffer Egeberg told Rappler.
If the information that Meta closes CrowdTangle is true, this perk may not be available for a long time.
As early as April 2021, Meta reportedly reassigned Crowdtangle staff to other divisions. More recently, Bloomberg reported that shutdown plans are still ongoing and some Facebook engineers have been tasked with killing the tool. CrowdTangle’s official internal shutdown process that was launched in February was initially put on hold due to Digital Services Act deliberations, according to Bloomberg.
Vincent said the reported move actually goes against the company’s commitments under the new Anti-Disinformation Code of Practice, which calls for more transparency about what anti-disinformation platforms are doing. “This decision would clearly make it less transparent for outside researchers to see what is happening on the platforms.”
In particular, the Code requires that safe design practices are in place to limit the spread of misinformation and ensure greater transparency of their recommendation systems, adapting them to limit the spread of misinformation.
The code also provides for the creation of a transparency center accessible to all citizens, and which “will make it possible to have a simple overview of the implementation of the measures of the code, ensuring transparency and regular updates of the data relevant”.
Signatories to the code include Meta, Google, Tiktok, Twitter, and other tech giants. The enhanced Code of Practice, together with the Digital Services Act and forthcoming legislation on transparency and targeting of political advertising, is a key part of the European Commission’s toolbox to combat the spread of misinformation.
For now, CrowdTangle is still accessible.
“If it is true that this tool (CrowdTangle) will be abandoned, it would make it much more difficult, if not impossible, for external researchers to verify whether Facebook is actually doing something and what its impact is on the circulation of misinformation. Science Feedback’s Vincent told Rappler.
Egebert from Faktisk added that it would hurt their work. “It will also make our fact-checking partnership with Meta less relevant, as access to CrowdTangle and our ability to influence CrowdTangle’s development is one of the key benefits.”
He goes further: “If the reasoning is what these rumors claim (too much transparency), it shows how little the leaders of Meta have learned or appreciated from our partnerships. That their own transparency through this tool also enables the critical press, should be welcomed. – Rappler.com