MASCA: LLM based-Multi Agents System for Credit Assessment
Summary
MASCA is an LLM-driven multi-agent system designed to enhance credit assessment by mirroring real-world decision-making processes. This framework employs a layered architecture where specialized LLM-based agents collaboratively handle sub-tasks, integrating contrastive learning for optimized risk and reward assessment. It also incorporates a signaling game theory perspective on hierarchical multi-agent systems, offering theoretical insights into their structure and interactions. Experimental results demonstrate MASCA's superior performance over baseline approaches, achieving 60% Accuracy, 83.33% Recall, and 73.33% F1 Score when combining gpt-4o and o3-mini. The system also includes a detailed bias analysis, revealing gender bias (65.22% accuracy for male vs. 58.26% for female) and ethnicity bias (African/Black 57.50%, Asian 52.50% accuracy, both below 60% ground truth).
Key takeaway
For financial institutions developing AI-driven credit assessment, consider adopting hierarchical LLM-based multi-agent systems like MASCA. This approach improves accuracy and adaptability over traditional methods by distributing tasks among specialized agents and integrating contrastive learning. Crucially, proactively analyze and mitigate inherent biases, as demonstrated by MASCA's findings on gender and ethnicity disparities, to ensure fair and compliant lending practices.
Key insights
MASCA employs a hierarchical LLM-based multi-agent system with contrastive learning to enhance credit assessment accuracy and address biases.
Principles
- Hierarchical multi-agent systems improve resilience and performance through specialized interactions.
- Division of labor among agents enhances explainability and error tracing.
- Signaling game theory provides a framework for strategic information sharing in agent hierarchies.
Method
The system uses a multi-tiered framework: Data Ingestion & Contextualization, Multidimensional Assessment (risk/reward teams), Strategic Optimization, and a final Decision Orchestrator.
In practice
- Implement specialized LLM agents for distinct credit assessment sub-tasks.
- Utilize contrastive learning to balance risk and reward evaluations.
- Perform detailed bias analysis on LLM-driven financial decision systems.
Topics
- LLM Agents
- Credit Assessment
- Multi-Agent Systems
- Financial Risk
- Bias Analysis
- Signaling Game Theory
Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Scientist, Research Scientist, AI Ethicist
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.