JU-NLP-PG at RAG4Reports 2026: Memory-Efficient Multilingual Report Generation with 4-bit Quantized LLMs

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, medium

Summary

The JU-NLP-PG system, developed for Task B on Multilingual Report Generation at RAG4Reports 2026, addresses the challenge of generating grounded, citation-bearing reports from a four million multilingual document corpus (English, Chinese, Russian, Arabic) given an English request. The core technical hurdle involved deploying a large retrieval corpus and a capable generative model on a two-GPU node with ≈29 GB RAM. To overcome this memory constraint, the system employs three key techniques: 4-bit NF4 quantization, which significantly shrinks the Large Language Model from ≈14 GB to ≈4 GB; the use of memory-mapped, chunked FAISS index construction built upon pre-computed multilingual-e5-large embeddings; and a strict model-loading order designed to prevent heap fragmentation. Additionally, the generated reports are structured using topic nuggets, directly aligning with the Auto-ARGUE evaluation signal.

Key takeaway

For Machine Learning Engineers deploying large Retrieval Augmented Generation (RAG) systems on resource-constrained hardware, this work demonstrates a viable path. You should consider implementing 4-bit NF4 quantization for your LLMs, which can reduce model size from ≈14 GB to ≈4 GB, alongside memory-mapped, chunked FAISS indices for multilingual corpora. Prioritize strict model-loading orders to prevent heap fragmentation, enabling complex multilingual report generation on systems with ≈29 GB RAM.

Key insights

Memory-efficient techniques enable large multilingual RAG systems on constrained hardware.

Principles

• Quantization drastically reduces LLM memory footprint.
• Memory-mapped indexing supports large retrieval corpora.
• Careful loading order prevents memory fragmentation.

Method

The system retrieves passages from a 4M multilingual corpus using FAISS, then generates reports structured by topic nuggets, employing 4-bit NF4 quantization and strict model-loading for memory efficiency on ≈29 GB RAM.

In practice

• Apply 4-bit NF4 quantization to LLMs.
• Use memory-mapped FAISS for large indices.
• Implement strict model loading sequences.

Topics

• Multilingual Report Generation
• Retrieval-Augmented Generation
• LLM Quantization
• FAISS Indexing
• Memory-Efficient AI
• multilingual-e5-large Embeddings

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Retrieval-Augmented Generation as a Bridge Between Classical NLP Pipelines and Modern LLM Systems — Retrieval-augmented generation (RAG) addresses the limitation of static knowledge in large language models (LLMs) by connecting them to external, up-to-date knowledge sources.
• Equity with Efficiency: An Empirical Study of Tokenizers for Multilingual Large Language Models — Cross-lingual fairness and tokenization efficiency are not fundamentally at odds in multilingual LLMs.
• Granite Embedding Multilingual R2: Open Apache 2.0 Multilingual Embeddings with 32K Context — Best Sub-100M Retrieval Quality — New IBM Granite multilingual embedding models offer superior retrieval quality and extended context for diverse language and code applications.
• Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents — Effective text chunking significantly improves RAG performance in low-resource, morphologically complex languages.
• Building a Semantic Search API: From Half a Million Documents to Millisecond Queries — Semantic search systems can achieve millisecond query times over large datasets using FAISS and Sentence Transformers.
• Linguistic Symptoms: Augmented Generation by Retrieval and Reasoning in LLMs under Portuguese-English Variation in Medical Contexts — Linguistic variation significantly impacts smaller LLMs in medical contexts, with larger models showing greater cross-language robustness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.