SMART: LLM-Augmented Hybrid Retrieval for Dynamic Product Ads

2026-07-25 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Information Retrieval, Natural Language Processing · Depth: Expert, quick

Summary

SMART (SeMantic-aware Adaptive ReTrieval) is a new system designed for Dynamic Product Ads (DPA) to efficiently retrieve relevant items from multi-million product catalogs. It addresses the challenge of balancing retargeting known interests with prospecting new categories. The system leverages a critical retrieval decomposition: rule-generated queries are used for retargeting on a lexical BM25 index, while LLM-generated queries handle prospecting on a dense ANN index. To manage the prohibitive inference costs of Large Language Models, SMART incorporates a lightweight quality gate that adaptively routes only about 10% of users, those who benefit most from semantic prospecting, to the LLM path. This approach achieves a 90% reduction in LLM costs while capturing significant semantic prospecting gains and maintaining competitive retargeting performance. A 2-week online A/B test at Snap demonstrated SMART improved the ad conversion rate by +27.6% compared to a strong embedding-based baseline.

Key takeaway

For MLOps Engineers deploying LLMs in large-scale retrieval systems, consider implementing a gated hybrid approach like SMART. This strategy allows you to achieve significant performance gains, such as a +27.6% ad conversion rate, by selectively applying LLM-powered semantic search to only ~10% of queries. This adaptive routing dramatically reduces LLM inference costs by 90%, making advanced retrieval economically viable for multi-million item catalogs without sacrificing retargeting effectiveness.

Key insights

SMART uses a gated hybrid retrieval combining lexical and LLM-based methods to boost ad conversions by +27.6% while reducing LLM costs by 90%.

Principles

• Decompose retrieval for retargeting (lexical) and prospecting (semantic).
• Route only high-value queries to expensive LLM paths.
• Lexical BM25 excels at retargeting known interests.

Method

SMART decomposes retrieval into rule-generated queries for lexical BM25 (retargeting) and LLM-generated queries for dense ANN (prospecting). A lightweight quality gate adaptively routes ~10% of users to the LLM path, managing LLM inference costs.

In practice

• Boost ad conversion rates by +27.6% for DPA.
• Reduce LLM inference costs by 90% in retrieval.
• Effectively balance retargeting and prospecting.

Topics

• Dynamic Product Ads
• Hybrid Retrieval
• Large Language Models
• LLM Inference Costs
• Ad Conversion Rate
• Information Retrieval

Best for: AI Product Manager, AI Architect, AI Engineer, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Howard University-AI4PC at SemEval-2026 Task 8: Query Reformulation and Dense-Lexical Retrieval Fusion for Multi-Turn Retrieval-Augmented Generation — A training-free hybrid RAG system combines multi-signal query reformulation and fused retrieval for competitive multi-turn performance.
• Decoupling Search from Reasoning: A Vendor-Agnostic Grounding Architecture for LLM Agents — Decoupling LLM search grounding from reasoning enables explicit control over retrieval, cost, and output behavior.
• Inside Target’s LLM-Based System for Semantic Matching in Marketing Forecast Pipelines — Target's system utilizes LLMs and semantic embeddings to automate and improve marketing campaign forecasting by matching historical data.
• MemBoost: A Memory-Boosted Framework for Cost-Aware LLM Inference — MemBoost reduces LLM inference costs by intelligently reusing answers and routing complex queries to a powerful oracle.
• Increase Recommendation Systems’ Precision with LLMs, Using Python — Optimize LLM-powered recommendation systems by combining cheap, high-recall filtering with expensive, high-precision reranking.
• LLM-Powered Deep Parsing for Industrial Inventory Search — LLM-powered deep parsing extracts structured data from messy industrial text for improved search and deduplication.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.