Identification of fake news in Portuguese: a look at the generalization of models

2026-04-12 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

A study investigated the generalization capabilities of BERTimbau and mBERT language models for fake news detection in Portuguese, specifically in cross-generalization scenarios where test data differed from training and validation data. Researchers fine-tuned these models using four Brazilian Portuguese corpora: Fake.br, Fakepedia, FakeRecogna, and FakeTrueBR. The findings confirmed that intra-base evaluations yielded high performance, while inter-base evaluations showed significant degradation in cross-generalization, despite the consistent objective of identifying fake news. Quantitatively, BERTimbau slightly outperformed mBERT, achieving an average accuracy of 71% and an f1-score of 67%, compared to mBERT's 69% accuracy and 64% f1-score.

Key takeaway

For research scientists developing fake news detection systems, you should prioritize rigorous cross-generalization testing beyond intra-base evaluations. The observed performance degradation in inter-base scenarios highlights the critical need for training data diversity and robust validation against real-world, varied datasets to ensure practical efficacy and avoid deploying models with limited real-world applicability.

Key insights

Language models for fake news detection show significant performance degradation in cross-generalization scenarios.

Principles

• Intra-base evaluations yield high performance.
• Inter-base evaluations show high degradation.

Method

Fine-tuning BERTimbau and mBERT on four Brazilian Portuguese corpora (Fake.br, Fakepedia, FakeRecogna, FakeTrueBR) to assess cross-generalization.

In practice

• Prioritize diverse training data.
• Validate models on unseen, real-world data.

Topics

• Fake News Detection
• Model Generalization
• BERTimbau
• mBERT
• Portuguese Language Models

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Lexical and Syntactic Characterization of Fake News in Portuguese Produced by Humans and by Machines — Machine-generated Portuguese fake news shows distinct lexical and syntactic patterns compared to human-produced content.
• Retrieval-Augmented Generation with Small Language Models for Fake News Detection — RAG-based fake news detection using web searches did not outperform traditional or deep learning methods.
• The 1.6 Billion People Large Language Models Still Can’t Understand — Large language models fail profoundly in many non-English languages, going beyond mere performance degradation to confident incorrectness.
• How easily can Russian propaganda fool AI models? A new benchmark finds out — A new benchmark quantifies AI language models' susceptibility to Russian propaganda narratives.
• Addressing Data Scarcity in Bangla Fake News Detection: An LLM-Based Dataset Augmentation Approach — LLM-based data augmentation significantly improves fake news detection in low-resource languages by generating diverse synthetic data.
• PorTEXTO: A European Portuguese Benchmark for Visual Text Extraction — Specialized multilingual data significantly improves European Portuguese OCR performance over model size or resolution.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.