A BART-based approach with hierarchical strategy for Vietnamese abstractive multi-document summarization

2026-06-19 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Advanced, long

Summary

A novel BART-based hierarchical approach addresses the Vietnamese multi-document abstractive summarization challenge from VLSP 2022. This method first condenses individual documents using a "golden summary"-driven strategy to ensure high correlation, then aggregates and summarizes them. It achieved a ROUGE2-F1 score of 0.2468 on the VLSP public test set, generating fluent and concise summaries. The research also significantly expanded available data by translating the Multinews dataset into Vietnamese, adding 50286 document clusters, which are now publicly available. This tackles the scarcity of Vietnamese multi-document summarization data and the input length limitations of models like BART for typical 2220-token clusters.

Key takeaway

For NLP Engineers developing multi-document summarization systems for low-resource languages like Vietnamese, consider adopting a hierarchical strategy. Your approach should involve an intermediate document shortening step guided by the "golden summary" to maintain information correlation. Additionally, augment your training data by translating large English datasets, such as Multinews, to overcome data scarcity. This method effectively addresses long input sequence challenges and improves summary fluency and conciseness.

Key insights

A hierarchical, BART-based approach for Vietnamese multi-document summarization uses "golden summaries" to guide document shortening, preventing information loss.

Principles

• Hierarchical summarization benefits from intermediate representations correlated with the final summary.
• Data scarcity in low-resource languages can be mitigated by translating large English datasets.

Method

A two-step process: 1) Train a seq-to-seq model to select top sentences from each document, guided by ROUGE1 scores against the golden summary. 2) Concatenate selected sentences and feed to a second seq-to-seq model for final summary generation.

In practice

• Translate large English datasets to augment low-resource language datasets.
• Use a two-step hierarchical model for long input sequences in multi-document summarization.

Topics

• Vietnamese NLP
• Multi-document Summarization
• Abstractive Summarization
• BARTPho
• Hierarchical Models
• Data Augmentation
• VLSP 2022

Code references

• ngoquanghuy99/Abmusu2022
• lupanh/VietnameseMDS

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• A BART-based approach with hierarchical strategy for Vietnamese abstractive multi-document summarization — A BART-based hierarchical strategy with golden summary-driven shortening improves Vietnamese multi-document abstractive summarization, achieving 0.2468 ROUGE2-F1.
• A Training-Free Mixture-of-Agents Framework for Multi-Document Summarization using LLMs and Knowledge Graphs — A training-free agent framework combines LLMs and knowledge graphs for effective multi-document summarization.
• Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents — Effective text chunking significantly improves RAG performance in low-resource, morphologically complex languages.
• The Essential Guide to Effectively Summarizing Massive Documents, Part 2 — Clustering and representative chunk selection enable scalable, structured summarization of large documents, significantly reducing token load.
• Translating Chinese with Data Science: StanfordNLP and Beyond — Integrating existing tools and broad IT understanding significantly accelerates complex translation tasks.
• A Comparative Study of Multi-document Summarization Techniques — The paper compares multi-document summarization techniques, contributing to NLP research.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.