How Experian Built AI Customer Support Agents that Boosted NPS
Summary
Experian Consumer Services, led by James Lynn, Head of AIM ML Innovation, successfully deployed AI customer support agents using Llama Index Core. The company adopted Llama Index in September 2023, citing its lighter orchestration footprint and strong community backing as key factors, especially within a regulated industry. This approach enabled Experian to modularize workflows, leading to increased Net Promoter Score (NPS) for products utilizing these agents. A significant performance improvement was observed, with Llama Index achieving a time to first token of 1 second compared to 8 seconds for other frameworks. The entire application was re-coded within Llama Index in a two-week sprint, and custom functionality was added to ensure high answer quality, including refusing answers with high retrieval distance and providing canned regulatory responses.
Key takeaway
For AI Architects building customer support solutions in regulated industries, your choice of orchestration framework significantly impacts development speed and answer quality. Consider lightweight, modular frameworks like Llama Index to achieve faster time to first token and enable rapid iteration. You should also prioritize frameworks with strong community support for long-term enterprise viability and integrate custom quality controls to meet regulatory requirements.
Key insights
Early adoption of lightweight, modular AI orchestration frameworks can significantly improve performance and customer satisfaction.
Principles
- Prioritize community-backed libraries for enterprise support.
- Modularize workflows for customizability and flexibility.
Method
Experian built AI agents by orchestrating models with Llama Index Core, focusing on modular workflows, and implementing quality controls like retrieval distance checks and canned regulatory responses.
In practice
- Evaluate AI frameworks for orchestration footprint.
- Implement answer quality controls for regulatory compliance.
Topics
- AI Customer Support
- Llama Index
- NPS Improvement
- Multi-Agent Systems
Best for: AI Architect, AI Engineer, Machine Learning Engineer, MLOps Engineer
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by LlamaIndex.