Disaster Question Answering with LoRA Efficiency and Accurate End Position

2026-02-26 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Public Safety & Security · Depth: Expert, extended

Summary

A new disaster-focused question answering system, optimized for Japanese contexts, has been developed to provide accurate information during emergencies. The system utilizes a hybrid architecture combining the cl-tohoku/bert-base-japanese-v3 model with Bidirectional Long Short-Term Memory (Bi-LSTM) and Enhanced Position Heads. It incorporates LoRA (Low-Rank Adaptation) for parameter efficiency, achieving 70.4% End Position accuracy and a 0.885 Span F1 score with only 5.7% of the total parameters (6.7M out of 117M). This architecture significantly outperforms BERT-base-only baselines by 25.1 percentage points in End Position accuracy. The system aims to address the scarcity of disaster-specific data and the risks of hallucinations in general-purpose models, providing a practical solution for real-world disaster response scenarios.

Key takeaway

For NLP Engineers developing emergency response systems, this work demonstrates that combining specialized BERT models with Bi-LSTM and LoRA optimization can achieve high accuracy and efficiency. You should consider adopting hybrid architectures and parameter-efficient fine-tuning for domain-specific, resource-constrained applications, especially where precise information extraction is critical for safety and rapid deployment.

Key insights

A LoRA-optimized Japanese BERT-Bi-LSTM hybrid system delivers high accuracy and efficiency for disaster-specific question answering.

Principles

• Hybrid architectures improve contextual and sequential understanding.
• Parameter-efficient methods enable deployment in resource-constrained environments.
• Domain-specific optimization outperforms general-purpose models for critical tasks.

Method

The system combines cl-tohoku/bert-base-japanese-v3 with Bi-LSTM and Enhanced Position Heads. LoRA optimization targets query/value matrices with rank-4 decomposition, freezing base parameters for efficient fine-tuning and precise answer span detection.

In practice

• Use LoRA for efficient fine-tuning of large models.
• Integrate Bi-LSTM for sequential understanding in procedural texts.
• Develop specialized position heads for precise span extraction.

Topics

• Disaster Question Answering
• LoRA Optimization
• Japanese NLP
• BERT-base
• Bidirectional LSTMs

Code references

• tk-yasuno/disaster-question-answer

Best for: NLP Engineer, AI Scientist, Research Scientist, AI Engineer, Machine Learning Engineer, AI Researcher

Related on AIssential

• Improving Answer Extraction in Context-based Question Answering Systems Using LLMs — Fine-tuning pre-trained LLMs on benchmark QA datasets significantly improves answer extraction accuracy and contextual understanding in QA systems.
• Multimodal Support System for Disaster Risk Management — NOAH is a multimodal AI system for disaster management, integrating NLP, speech, and vision via WhatsApp for Brazilian cities.
• Turkish Sentiment Analysis (Keras LSTM + BERT) — A hybrid NLP pipeline combines custom LSTMs and pre-trained BERT for robust Turkish sentiment analysis with real-time data acquisition.
• Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language — A compact 135M parameter LLM for Bangla achieves 1B-tier performance by efficiently merging tokens, overcoming low-resource script challenges.
• An Efficient Approach for Answering Not Readily Attainable Questions for RAG-based Applications — This paper proposes an efficient approach for RAG systems to answer questions not readily found in retrieved data.
• Comparing BERT Sentence-Pair Classification and Few-Shot LLM Prompting for Detecting Threat and Solution Framing in German Climate News — Fine-tuned BERT models outperform few-shot LLMs for German climate news framing detection, especially with sentence context.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.