PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models

2026-07-03 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Information Retrieval · Depth: Expert, quick

Summary

PORTS, a novel odds ratio preference optimization method, addresses the challenge of Large Language Models (LLMs) struggling with extensive tool collections by improving tool selection accuracy. Existing retrieval methods often misalign with tool-calling LLMs due to separate training. PORTS fine-tunes a retriever using a perplexity-inspired preference signal from a frozen LLM, optimizing the correlation between selection probabilities and downstream performance while enforcing a contrastive semantic loss on documentation strings. This approach significantly enhances tool selection accuracy, demonstrated across six datasets, two encoder models, and three LLMs. With low computational demands, PORTS facilitates generalization to new queries and tools, proving valuable for practical applications with evolving toolsets.

Key takeaway

For Machine Learning Engineers integrating LLMs with extensive tool libraries, PORTS offers a low-computational method to significantly enhance tool selection accuracy and generalization. You should consider implementing this preference optimization approach to ensure your LLMs effectively utilize dynamic and expanding toolsets, reducing latency and improving task completion across diverse applications.

Key insights

PORTS aligns tool retrievers with LLMs using perplexity-based preference optimization for improved selection accuracy.

Principles

• Align retrievers with LLMs via preference optimization.
• Utilize perplexity as a preference signal.
• Combine performance correlation with semantic loss.

Method

PORTS fine-tunes a retriever using an odds ratio preference optimization, correlating selection probabilities with downstream LLM performance, and applies a contrastive semantic loss on documentation strings.

In practice

• Improve LLM tool selection accuracy.
• Generalize tool selection to new queries.
• Manage evolving toolsets efficiently.

Topics

• Large Language Models
• Tool Selection
• Information Retrieval
• Preference Optimization
• Machine Learning
• Contrastive Learning

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Understanding the Behaviors of Environment-aware Information Retrieval — LLMs can learn to adapt query strategies for diverse retrievers using RL, revealing distinct optimal query styles.
• Self-Evolution for Multi-Turn Tool-Calling Agents via Divergence-Point Preference Learning — The paper combines ToolGraph with DPO, using divergence points for preference learning to enhance multi-turn tool-calling agents.
• The Model You Love Is Probably Just the One You Use — Developer preference for LLMs is often driven by familiarity and integration ease, not objective superiority.
• Learning to Route LLMs from Implicit Cost-Performance Preferences via Meta-Learning — MetaRouter optimizes LLM routing by meta-learning implicit user cost-performance preferences via contextual bandits.
• Token-weighted Direct Preference Optimization with Attention — Using an LLM's intrinsic attention to weight tokens during preference optimization significantly improves alignment and performance.
• AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning — LLM agents can achieve dynamic tool selection and superior generalization by optimizing tool preferences via Plackett–Luce ranking.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.