When Rules Learn: A Self-Evolving Agent for Legal Case Retrieval

2026-06-15 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A self-evolving framework has been proposed to enhance BM25 for legal case retrieval, addressing challenges posed by complex legal language and the need for precise lexical alignment. This framework utilizes an LLM-based agent equipped with an automatic evaluation environment. The agent iteratively creates query rewriting rules, plans validation experiments for rule combinations, and eliminates ineffective rules based on historical feedback, all without any parameter training. Evaluated on the Chinese legal case retrieval benchmark LeCaRD-v2, the proposed method significantly outperforms non-evolutionary baselines, including human-designed rules and greedy rule selection. Its effectiveness is particularly pronounced when powered by a high-capacity core LLM. Findings indicate that the LLM's capacity to leverage previous experimental results and its intrinsic knowledge for rule elimination are crucial to the self-evolution process.

Key takeaway

For NLP Engineers developing legal case retrieval systems, this research suggests you should explore self-evolving, LLM-based query rewriting frameworks to significantly enhance BM25 performance. You can achieve superior lexical alignment and retrieval accuracy without extensive parameter training by implementing an automatic evaluation environment for iterative rule refinement. Consider integrating a high-capacity core LLM to maximize the effectiveness of rule creation and elimination, moving beyond static or human-designed rule sets.

Key insights

An LLM-based agent can self-evolve query rewriting rules to enhance BM25 for legal case retrieval, leveraging an automatic evaluation environment without parameter training.

Principles

• LLMs can refine rules via self-evolution.
• Historical feedback is key for rule elimination.
• Iterative rule creation surpasses static baselines.

Method

An LLM-based agent iteratively creates query rewriting rules, plans validation experiments over rule combinations, and eliminates ineffective rules based on historical feedback within an automatic evaluation environment.

In practice

• Enhance BM25 with LLM-driven query rewriting.
• Implement automatic evaluation for rule refinement.
• Prioritize high-capacity LLMs for optimal results.

Topics

• Legal Case Retrieval
• Query Rewriting
• Large Language Models
• BM25
• Self-Evolving Agents
• LeCaRD-v2

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• RAG's Evolution: From Simple Retrieval to Agentic AI — Search evolution progresses from keyword matching to semantic understanding and autonomous, adaptive retrieval.
• Development and Evaluation of a Hybrid Information Retrieval System Applied to the Brazilian Legal Domain — Hybrid IR systems combining traditional and Transformer models can overcome long-text limitations in specialized domains.
• The State of Information Retrieval in 2026 — Information retrieval has evolved into a reasoning problem, integrating LLMs and agentic search for complex, context-aware query processing.
• SkillPager: Query-Adaptive Intra-Skill Navigation via Semantic Node Retrieval — Typed semantic node retrieval significantly reduces LLM prompt tokens while maintaining high context sufficiency for skill-based agents.
• VLegal-Bench: Cognitively Grounded Benchmark for Vietnamese Legal Reasoning of Large Language Models — VLegal-Bench offers a cognitively-grounded benchmark for LLMs in Vietnamese civil law, revealing specialized models outperform general ones on complex legal reasoning.
• Parthenon Law: A Self-Evolving Legal-Agent Framework — Self-evolving legal agents improve performance by learning from failures without modifying core LLM weights.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.