01 / Key message
DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts
A modular, zero-shot system that verifies open-domain image-text claims by dynamically retrieving and reasoning over multimodal evidence.
Dynamic multimodal evidence
02 / Method
How it works
Plan the check
Interpret the image-text claim and decide which evidence and specialist tools the investigation requires.
Retrieve evidence
Search textual and visual sources dynamically, expanding the investigation when the current evidence is insufficient.
Build the report
Evaluate the collected evidence, infer a verdict, and present the reasoning in a structured multimodal report.
03 / Abstract
Abstract
The proliferation of disinformation demands reliable and scalable fact-checking systems that can handle both text and images. DEFAME is a modular, zero-shot multimodal large-language-model pipeline for open-domain claim verification. Its six-stage process dynamically selects tools and search depth to retrieve, evaluate, and integrate textual and visual evidence, then produces a structured fact-checking report. Unlike systems that are text-only or rely on parametric knowledge, DEFAME performs the complete verification process with multimodal claims and evidence. It establishes new state of the art across VERITE, AVeriTeC, MOCHEG, and the temporally challenging ClaimReview2024+ benchmark.
04 / Contributions
What this adds
- 01
End-to-end multimodal verification
Handles images in both claims and retrieved evidence while producing a structured, evidence-grounded report.
- 02
Dynamic investigation
Lets the model choose tools and search depth instead of applying one fixed retrieval recipe to every claim.
- 03
Temporally robust evaluation
Introduces ClaimReview2024+, whose claims postdate the backbone model's knowledge cutoff and reduce the value of memorization.
05 / Citation
Citation
Braun, T., Rothermel, M., Rohrbach, M., & Rohrbach, A. (2025). DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts. Proceedings of the 42nd International Conference on Machine Learning, 267, 5383–5417.
BibTeX
@inproceedings{braun2025defame,
title = {{DEFAME}: Dynamic Evidence-based {FA}ct-checking with Multimodal Experts},
author = {Tobias Braun and Mark Rothermel and Marcus Rohrbach and Anna Rohrbach},
booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
volume = {267},
pages = {5383--5417},
year = {2025},
url = {https://proceedings.mlr.press/v267/braun25b.html}
}