Amortizing Maximum Inner Product Search with Learned Support Functions

2026-03-09 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Advanced, quick

Summary

A new learning-based approach called amortized Maximum Inner Product Search (MIPS) has been proposed to predict MIPS solutions directly using neural networks. This method aims to amortize the computational cost of matching queries from a fixed distribution to a fixed set of keys. The core idea leverages the MIPS value function, which is equivalent to the support function of the key set, known for being convex and 1-homogeneous. Two main approaches are introduced: SupportNet, an input-convex neural network that models the support function directly, recovering the optimal key via gradient computation; and KeyNet, a vector-valued network that regresses the optimal key from the query, bypassing gradient computation during inference. Training SupportNet involves score regression and gradient matching losses with homogenization wrappers, while KeyNet uses a score consistency loss derived from Euler's theorem. Experiments demonstrate high match rates and potential for database compression.

Key takeaway

For research scientists working on efficient similarity search in machine learning, this amortized MIPS approach offers a novel way to significantly reduce computational costs. You should explore integrating SupportNet or KeyNet into your MIPS pipelines, especially when dealing with fixed key sets and known query distributions, to achieve higher match rates and potentially compress your databases.

Key insights

Amortized MIPS uses neural networks to predict optimal keys by modeling the support function or directly regressing keys.

Principles

• MIPS value function is a support function.
• Support functions are convex and 1-homogeneous.
• Gradients of support functions yield optimal keys.

Method

Train SupportNet with score regression and gradient matching, or KeyNet with score consistency loss, to predict MIPS solutions directly from queries.

In practice

• Compress databases with specific query distributions.
• Accelerate MIPS in machine learning subroutines.
• Utilize input-convex neural networks for support functions.

Topics

• Maximum Inner Product Search
• Amortized Learning
• Support Functions
• Input-Convex Neural Networks
• Gradient Matching

Best for: Research Scientist, AI Researcher, AI Scientist, Machine Learning Engineer

Related on AIssential

• Amortizing Maximum Inner Product Search with Learned Support Functions — Amortized MIPS uses neural networks to predict optimal keys, exploiting the support function's convex and homogeneous properties.
• One Operator for Many Densities: Amortized Approximation of Conditioning by Neural Operators — Neural operators can universally approximate probabilistic conditioning, enabling amortized inference across diverse joint distributions.
• Large-Scale Data Parallelization of Product Quantization and Inverted Indexing Using Dask — Combining PQ, Inverted Indexing, and Dask enables efficient, accurate large-scale ANN search.
• Minibatch Selection via Partition Matroid Constrained Gradient Matching — PartitionSel optimizes LLM fine-tuning on heterogeneous data by balancing domain coverage and convergence via gradient-matching and matroid constraints.
• How does loss functions work in PINN? [D] — Neural networks learn from loss function gradients, not absolute values, with coefficients shaping training sensitivity.
• Largest empty shapes — Finding largest empty shapes in point sets requires diverse optimization models depending on the shape.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.