The ‘brownie recipe problem’: why LLMs must have fine-grained context to deliver real-time results

2026-02-04 · Source: VentureBeat · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Data Science & Analytics · Depth: Intermediate, short

Summary

Instacart CTO Anirban Kundu highlights the "brownie recipe problem," where large language models (LLMs) struggle with fine-grained, real-time context in dynamic ordering systems. While LLMs excel at reasoning, they must integrate real-world state, user preferences, and logistical constraints to be truly assistive, such as understanding ingredient availability, organic vs. regular options, and delivery geography to prevent spoilage. Instacart addresses this by splitting processing: a foundational LLM handles intent and categorization, then routes data to smaller language models (SLMs) for catalog context (product relationships, substitutions) and semantic understanding (e.g., "healthy snacks for children"). This modular approach avoids unmanageable monolithic models and ensures sub-second response times, crucial for user retention.

Key takeaway

For CTOs and VPs of Engineering building real-time, context-aware AI systems, your teams should prioritize modular architectures using specialized smaller models and microagents. This approach, exemplified by Instacart, effectively manages latency and integrates complex real-world state and user preferences without creating unmanageable monolithic LLMs, thereby improving user experience and system reliability.

Key insights

LLMs require fine-grained, real-time context and modular architectures to deliver effective, low-latency assistance in dynamic systems.

Principles

• Balance reasoning with real-world state.
• Modular agents outperform monolithic systems.
• Latency is critical for user retention.

Method

Instacart uses a two-stage processing approach: a foundational LLM for intent and categorization, followed by specialized SLMs for catalog context and semantic understanding, integrated with microagents for external systems.

In practice

• Split LLM processing into specialized chunks.
• Implement microagents for diverse external platforms.
• Utilize protocols like OpenAI MCP for agent integration.

Topics

• Real-time Contextual AI
• Hybrid LLM Architectures
• AI Agent Systems
• E-commerce AI Applications
• Model Context Protocols

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Engineer, Machine Learning Engineer, AI Architect

Related on AIssential

• How AI Agents Use MCPs to Deliver Stronger Results Than Standalone LLMs — AI Agents leveraging Model Context Protocol (MCP) transform LLMs from conversational assistants into action-oriented problem solvers by integrating external systems.
• LLM Research Papers: The 2026 List (January to May) — LLM research in early 2026 emphasizes hybrid architectures and long-context efficiency beyond simple scaling.
• Ralph Wiggum (and why Claude Code's implementation isn't it) with Geoffrey Huntley and Dexter Horthy — Effective LLM agent design requires deterministic context management and human oversight, not autonomous "in-loop" operation.
• Large Language Models in Transportation Systems Management and Operations: From Text Reasoning to Multi-modal Decision Support — LLMs and MM-LLMs offer a new mechanism for integrating diverse transportation data into operator-facing decision support systems.
• No AI Agent Orchestration Needed? What? — Modern frontier LLMs can self-orchestrate complex procedural tasks more effectively via in-context prompting than with external agent frameworks.
• Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems — Latent communication in LLM multi-agent systems reduces cost and information loss by exchanging continuous representations directly.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by VentureBeat.