Think Harder and Don't Overlook Your Options: Revisiting Issue-Commit Linking with LLM-Assisted Retrieval

2026-07-01 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning · Depth: Advanced, extended

Summary

This study revisits issue-commit linking, a critical aspect of software traceability, by evaluating various retrieval and reranking techniques. Researchers assessed established methods like BTLink, EasyLink, FRLink, RCLinker, and Hybrid-Linker, alongside modern deep learning and large language models (ChatGPT, Qwen, Gemma, Llama). The evaluation focused on efficiently retrieving relevant commits and then refining their ranking. Key findings indicate that dense retrieval methods outperform sparse approaches in identifying relevant commits, and combining dense and sparse retrieval (e.g., via Reciprocal Rank Fusion) can improve recall. Crucially, traditional machine learning-based reranking techniques, such as RCLinker, achieved higher performance than LLM-based approaches, suggesting that simpler models warrant careful consideration before adopting computationally expensive LLM solutions for large-scale issue-commit linking. The study used three datasets, including BTLink, EasyLink, and a new Apache dataset, with a hybrid temporal window capturing 97% of true links.

Key takeaway

For Machine Learning Engineers optimizing software traceability, prioritize evaluating traditional machine learning models and hybrid retrieval strategies before committing to expensive LLM-based solutions. Your team can achieve robust issue-commit linking performance by combining dense and sparse retrieval methods with a model like RCLinker, which leverages contextual metadata. This approach often yields superior accuracy and significantly lower computational costs, making it more practical for large-scale, real-time integration.

Key insights

Dense retrieval and traditional ML models often outperform LLMs for issue-commit linking, emphasizing simpler solutions.

Principles

• Dense retrieval methods generally outperform sparse methods.
• Combining dense and sparse retrieval improves recall.
• Simpler ML models can exceed complex LLM performance.

Method

A two-stage pipeline involves temporal filtering to narrow candidate commits, followed by retrieval (dense, sparse, or hybrid) and then reranking using ML or LLM-based techniques.

In practice

• Adopt a hybrid temporal window (1 year post-creation + 30 days around closure).
• Combine dense and sparse retrieval using Reciprocal Rank Fusion.
• Benchmark traditional ML models like RCLinker before LLMs.

Topics

• Issue-Commit Linking
• Software Traceability
• Retrieval Methods
• Reranking Techniques
• Large Language Models
• Machine Learning Models
• Reciprocal Rank Fusion

Code references

• spotify/annoy
• facebookresearch/faiss
• dorianbrown/rank_bm25

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora — Existing RAG benchmarks misrepresent real-world performance due to high document redundancy and similarity in enterprise corpora.
• Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation — Document-side chunk aggregation with local position encoding prevents evidence dilution in long-document dense retrieval.
• The State of Information Retrieval in 2026 — Information retrieval has evolved into a reasoning problem, integrating LLMs and agentic search for complex, context-aware query processing.
• An LLM as arbiter in RAG retrieval: picking the right candidate with reasons — An LLM arbiter, given structured candidate briefs, can rank RAG results with auditable reasons, surpassing score fusion.
• Anchor Detection for RAG: Parallel Detectors, Then One LLM Call at the End — Enterprise RAG retrieval combines parallel keyword and embedding detectors, aggregating results for a single, auditable LLM ranking.
• Rethinking Semantic–Collaborative Integration in Recommenders, Efficient Token-Aware Clustering for Multivector Retrieval, and More! — Effective information retrieval systems benefit from managing complementarity, optimizing indexing, and distilling LLM utility.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.