Why intent prediction needs more than an LLM
Summary
Yobi, a behavioral AI company, is developing foundation models specifically designed for predicting future behavior across various domains like ad tech and marketing. CTO Frank Portman argues that large language models (LLMs) are inherently ill-suited for complex intent prediction and decision-making under uncertainty due to their inductive bias towards next-token prediction on language data. Yobi's approach utilizes large-scale transformers and graph models trained on proprietary, sensitive, and often non-textual behavioral data, which can involve three orders of magnitude more "tokens" than typical language models. The company focuses on creating a "broadly predictive" base representation of future actions, enabling applications beyond initial ad placement to include broader personalization, marketing, fraud detection, and risk management. Their inference stack employs significant pre-computation and batching to handle millions of queries per second, prioritizing model-based solutions over complex heuristics.
Key takeaway
For AI Engineers or Directors of ML building predictive systems, recognize that general-purpose LLMs may not suffice for nuanced intent prediction or decision-making with behavioral data. You should evaluate specialized behavioral AI foundation models that leverage proprietary data and inductive architectures. Consider integrating such models as a dedicated decision layer within agentic systems, rather than relying solely on LLMs' inherent biases. Prioritize robust inference stacks with pre-computation and batching, and explore privacy-preserving machine learning techniques to build trust and manage sensitive consumer data effectively.
Key insights
LLMs' language-focused inductive bias makes them suboptimal for nuanced behavioral intent prediction and decision-making.
Principles
- Behavioral AI requires specialized foundation models beyond language.
- Inductive architectures are crucial for dynamic, high-cardinality behavioral data.
- Prioritize model-based solutions over complex heuristic walls for prediction.
Method
Train large-scale transformers and graph models on proprietary, sensitive, high-cardinality behavioral data to create a "broadly predictive" base representation for future actions.
In practice
- Implement pre-computation and batching for high-throughput inference.
- Explore privacy-preserving ML (e.g., homomorphic ML) for sensitive data.
Topics
- Behavioral AI
- Intent Prediction
- Foundation Models
- Large Language Models
- Graph Neural Networks
- Privacy-Preserving ML
- Ad Tech
Best for: AI Product Manager, AI Engineer, Machine Learning Engineer, Director of AI/ML
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Stack Overflow Blog.