Agents need more than a chat - Jacob Lauritzen, CTO Legora
Summary
Legora, a vertical AI company for law firms with over 1,000 customers, addresses the challenges of complex, long-running AI agents, particularly the "context rot" that leads to agents forgetting prior instructions. The company highlights a shift in the economics of production, where planning and reviewing work, rather than execution, are now the primary bottlenecks. They introduce "the verifier's rule," stating that AI will solve tasks that are both solvable and easy to verify, noting that legal tasks like contract writing are solvable but difficult to verify. Legora proposes strategies for effective human-agent collaboration, focusing on increasing trust through task decomposition, guardrails, and proxy verification, and increasing control through planning, skills, and elicitation. They advocate for high-bandwidth, persistent interfaces beyond traditional chat for complex agent interactions, such as collaborative documents or tabular reviews, to facilitate better human oversight and judgment.
Key takeaway
For AI Architects designing vertical AI solutions, recognize that traditional chat interfaces are insufficient for complex agent collaboration. Prioritize developing high-bandwidth, persistent interfaces like collaborative documents or structured tabular reviews. This approach will enhance human control and trust, mitigate context rot, and enable more effective human-in-the-loop verification, ultimately improving the reliability and adoption of your AI agents.
Key insights
Complex AI agents require high-bandwidth interfaces and structured human collaboration to overcome context rot and verification challenges.
Principles
- If a task is solvable and easy to verify, AI will solve it.
- Planning and reviewing are the new bottlenecks in AI production.
- Agents are not humans; do not constrain them to human language interfaces.
Method
Increase agent trust by decomposing tasks, using proxy verification, and adding guardrails. Increase human control via planning, encoding skills for contingencies, and eliciting user input for unknown scenarios.
In practice
- Decompose complex tasks into verifiable sub-tasks.
- Implement guardrails to limit agent actions and increase trust.
- Use persistent, high-bandwidth interfaces (e.g., documents, tables) for agent collaboration.
Topics
- Complex AI Agents
- Vertical AI
- Agent-Human Collaboration
- Verifier's Rule
- High Bandwidth Interfaces
Best for: AI Engineer, AI Architect, AI Product Manager
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by AI Engineer.