Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval

2026-07-24 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, FinTech & Digital Financial Services · Depth: Expert, quick

Summary

DS@GT's submission to FinMMEval 2026 Task 1 addresses multilingual financial exam question answering across English, Spanish, Greek, Chinese, and Hindi. The system employs a retrieval-augmented pipeline built on LangGraph, which detects query language and retrieves semantically relevant exemplars from a 30,209-entry multilingual knowledge base using BGE-M3 embeddings and FAISS indexing. Answers are scored via Retrieval-Augmented Direct Scoring (RADS), which reads next-token log-probabilities over candidate option letters instead of generating free-form text. For low-resource languages, the system fuses per-language and cross-lingual retrieval indices using weighted Reciprocal Rank Fusion. Model selection is language-routed, utilizing Qwen3-14B for Arabic, Chinese, and Hindi; Qwen2.5-14B for English; and Llama-3.1-8B for Greek. Empirical ablations revealed significant language-asymmetric performance gaps, with chain-of-thought prompting degrading Greek accuracy from 90.7% to 20.9% and Qwen3's default thinking mode collapsing Arabic RADS performance.

Key takeaway

For NLP Engineers developing multilingual financial QA systems, you should prioritize language-aware design. Implement explicit language detection and route queries to empirically validated language-specific models like Qwen3-14B or Llama-3.1-8B. Consider Retrieval-Augmented Direct Scoring (RADS) for answer selection, as free-form generation or chain-of-thought prompting can significantly degrade accuracy in certain languages, such as Greek, where performance dropped from 90.7% to 20.9%.

Key insights

Effective multilingual financial QA requires language-aware retrieval, model routing, and deliberate scoring strategies.

Principles

• Chain-of-thought can degrade accuracy in specific languages.
• Default "thinking modes" may harm direct scoring performance.
• Multilingual RAG benefits from fused per-language indices.

Method

The system uses LangGraph for language detection, BGE-M3 embeddings with FAISS for retrieval from a 30,209-entry knowledge base, and Retrieval-Augmented Direct Scoring (RADS) for answer selection.

In practice

• Implement language-specific model routing for diverse LLMs.
• Test direct option scoring (RADS) over free-form generation.
• Fuse retrieval indices for low-resource language support.

Topics

• Multilingual QA
• Financial NLP
• Retrieval-Augmented Generation
• Language Model Routing
• Direct Option Scoring
• BGE-M3 Embeddings

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Overview of FinMMEval 2026 Task 2: Multilingual Financial Short-Answer Question Answering — FinMMEval 2026 Task 2 evaluates multilingual financial short-answer QA across diverse languages and difficulty tiers.
• Qwen Goes Brrr: Off-the-Shelf RAG for Ukrainian Multi-Domain Document Understanding — Effective RAG for document understanding prioritizes structural context and answer-aware relevance.
• Linguistic Issues in Computational Linguistics for Non-English Languages — English-centric LLM design creates systematic linguistic weaknesses for non-English languages across multiple levels.
• QQ: A Language Metadata Toolkit for Multilingual NLP — QwanQwa unifies disparate language metadata and identifiers into a traversable graph for scalable multilingual NLP.
• bLLeQA: Benchmarking LLMs for Grounded Legal Question-Answering in French and Dutch — RAG systems for legal Q&A in French and Dutch face significant challenges, especially in generation quality and source refusal.
• Overview of FinMMEval 2026 Task 1: Multilingual Financial Multiple-Choice Question Answering — FinMMEval 2026 Task 1 benchmarks multilingual financial QA, highlighting effective techniques for robust cross-language reasoning.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.