The FactDeMice Project: AI as a Tool for Journalists and Businesses, Not a Arbiter of Truth

Since last year, an interesting international project called FactDeMice has been underway at the Faculty of Information Technology. Broadly speaking, it focuses on the field of fact-checking—verifying facts based on textual evidence, detecting fake reviews, and automatically extracting misleading claims. The principal investigator is Dr. Martin Fajčík from the Institute of Computer Graphics and Multimedia and the KNOT@FIT research group. Fajčík has long been engaged in research on machine learning for information retrieval—the project funded by the Technology Agency of the Czech Republic is thus a natural continuation of his professional journey.

Disinformation in comments? Context is king

Even if we consider only the brief description of the project provided above, its potential practical impact and societal relevance are evident. Online communication (from news articles to discussion forums to product reviews) has become a crucial arena where public opinion is shaped today. However, these channels are also targets of abuse: disinformation systematically disrupts the functioning of markets, undermines public trust, and ultimately threatens democratic processes. The motivation behind the FactDeMice project is clear. Yet caution and modest expectations are warranted. “The fundamental problem in the world of fact-checking is the very definition of truth itself. And even if we manage to pin it down, an even greater challenge is convincing people that this is indeed the case. A major turning point in our field was the introduction of the Community Notes feature on what was then Twitter. Currently, the effort to provide context has become something of a gold standard,” comments Fajčík on the ambitions of this and other projects.

The first content pillar of the project and, at the same time, the primary focus of the research by the Brno experts involved in the project is the area of misinformation in comments, occurring, for example, on news platforms. Specifically, this involves their identification, verification, and subsequent addition of context. “Essentially, we’re examining how comments would function if community notes were added to them. In the Czech Republic or Slovakia, this mechanism isn’t widely used. The Seznam Zprávy website began adding additional context to comments in 2023, but these are fact-checks from a pre-prepared database, not context provided on demand based on a current comment. That’s where we’re heading. “It occurred to us: What if we provided online platforms with a system that would automatically help moderate misinformation?” comments the project’s principal investigator on one of the project’s key goals. For the researchers involved in the project, large language models serve as the tool for this type of work with comments. It was precisely the rapid development in the field of artificial intelligence that provided one research motivation for their work: In August 2025, AI agents added community notes to the X network for the first time. “So we are now trying to create a benchmark that evaluates systems that generate community notes. That is where our Brno expertise lies,” says Fajčík, commenting on the structure of the project’s goals.

Fake reviews are also in the crosshairs

FactDeMice is a bilateral project. The second research partner is the Taiwanese university National Taipei University of Technology. The project also has a business partner, the company Lingea. “The initiative for the project came from us; we met with the Taiwanese side during delegation talks and subsequently contacted our colleagues at Lingea. Taiwan strongly supports research projects with foreign partners.” National Taipei University of Technology provides expertise in the second research pillar. This is the detection of fake reviews. The Taiwanese side focuses on reviews of technological products; in the second phase, the focus will shift to misleading restaurant reviews in discussion forums. And once again, we can point to a specific historical event that motivates this research: In 2013, Samsung had to pay a fine for funding a campaign of fake reviews of its competitor’s products.

The goal of the project in the second pillar? Identifying fake reviews based on specific indicators. This is all done through training machine learning models. Subsequently, scenarios will be designed. “We will have a system capable of identifying fake reviews. Then we would like to simulate similar attacks ourselves—that is, design entire artificial scenarios: Here we will define the actors and motivations, generate reviews using LLM, and try to determine whether the systems we have designed are capable of identifying these reviews as well,” Fajčík comments on the plan. Then, in the final step, comes analysis of existing data (reviews), which experts are also collecting on the Czech side, in collaboration with Lingea.

Benchmarks and a software tool

The project’s results include, on the one hand, previously nonexistent datasets serving as benchmarks for the community. And, on the other hand, a specific software product. Returning to the first pillar, in which FIT VUT plays a leading role, the benchmark regarding comments and the resulting software solution proceed in four steps (see also the infographic):

1. Assessment of the meaningfulness of comment verification—in terms of its social relevance, whether it involves, for example, mere vulgarities or a purely subjective opinion of the author, etc. Only one-third of comments on the Czech and Slovak platforms included in the research pass through this first filter. Of course, this also depends on the topic—whether it involves, for example, a tabloid like Blesk.cz or a technical domain such as Lupa.cz.
2. If a comment is identified that warrants verification, we must extract the claim from it that we wish to verify. The problem here lies in the following condition that must be met: The claim in the comment must be understandable on its own, without the original article or subsequent or preceding reactions. Again, this is a relatively fine-meshed content filter.
3. In the third step, records are searched for in web sources to determine the exact location. Martin Fajčík emphasizes that the system should not assess the truthfulness or falsity of a claim; it relies on the frequency of references to the source—in the future, a system administrator or user could set the source’s rating as the decisive factor (one might recall the existing Reuters rating agency).
4. The final step is then to generate final structured summaries for users.

Reviews present a specific challenge. For the most part, they are not factual. However, they can still be false, and that is what interests researchers. “In their case, the signs of falsity are only superficial. Colleagues from Taipei University of Technology annotate signs of falsity that help them determine the authenticity of a review,” Fajčík describes the situation.

An area that is much discussed, but lacks solutions

For Martin Fajčík, the FactDeMice project is not a step outside his existing research path. Since 2018, he has been working on systems that retrieve information and answer questions (Open Domain Question Answering systems). “Most of my doctoral work was based on this topic; after that, I went to Switzerland for a while to work at the Idiap research institute. There, we had a project requiring similar technologies, which focused on migration crises, fact-checking, and retrieving evidence from data provided by the migrants themselves. The primary goal was no longer to answer questions, but to look for what supports a given claim and what refutes it, or rather, whether it can be verified at all. Today we know that what matters in fact-checking isn’t the verdict, but the ‘story,’ the context,” Fajčík reflects on his early research days. Of course, one cannot avoid asking about his perception of the connection between the current project and the society-wide problem of disinformation. “To be honest: I’m interested in the system and the mathematics behind it. Disinformation is part of public discourse, and context can help identify it. But it depends on the users and readers themselves how they handle the context and whether they are capable of doing so.”

Martin Fajčík also points out another fundamental aspect of working with disinformation: “It’s an area that’s frequently discussed in the Czech Republic and Slovakia, but I don’t see anyone coming up with any concrete technical solutions. And yet, such solutions do exist in the international arena. I view providing context for comments as a democratic tool.” The FactDeMice project, in which doctoral students from our faculty are participating alongside Fajčík, is scheduled to conclude at the end of 2027. We are, of course, curious about its findings and wish the researchers every success.