ChatSVA: Bridging SVA Generation for Hardware Verification via Task-Specific LLMs

2025-07-14 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation, Hardware Design & Verification · Depth: Expert, extended

Summary

ChatSVA is an end-to-end multi-agent system designed to automate SystemVerilog Assertion (SVA) generation for hardware verification, addressing challenges of low functional accuracy and data scarcity in Large Language Model (LLM) applications. The system employs a multi-agent framework, with its core AgentBridge platform systematically generating high-purity, domain-specific datasets for few-shot scenarios. Evaluated on 24 RTL designs, ChatSVA achieved a 98.66% syntax pass rate and a 96.12% functional pass rate, generating 139.5 SVAs per design with 82.50% function coverage. This performance represents a 33.3 percentage point improvement in functional correctness and an over 11x enhancement in function coverage compared to the previous state-of-the-art (SOTA), AssertLLM. An online service for ChatSVA is publicly available.

Key takeaway

For research scientists focused on hardware verification, ChatSVA demonstrates that decomposing complex tasks into modular sub-tasks and generating high-purity, domain-specific training data are critical for achieving high functional accuracy and coverage with LLMs. You should consider adopting multi-agent architectures and robust data synthesis pipelines to overcome limitations of monolithic LLM approaches in specialized engineering domains.

Key insights

ChatSVA significantly improves SVA generation by decomposing reasoning and synthesizing high-purity, domain-specific data for LLMs.

Principles

• Outputs must be functional subsets of inputs.
• Inputs must originate from verified "golden" datasets.
• Generated outputs must be rigorously verifiable.

Method

ChatSVA uses a four-stage LLM pipeline (SpecWiz, Feature Generator, Checkpoint Generator, SVA Generator) to decompose SVA generation. AgentBridge synthesizes high-purity datasets via a self-improving closed loop with directional information constraints, ground truth provenance, and output verifiability.

In practice

• Use multi-agent frameworks for complex reasoning tasks.
• Synthesize domain-specific data to overcome scarcity.
• Employ reverse-generation for robust data validation.

Topics

• SystemVerilog Assertions
• Hardware Verification
• Large Language Models
• Multi-agent Systems
• Data Synthesis

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Hardware Engineer

Related on AIssential

• Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation — Automated testbench generation enables data-efficient LLM fine-tuning for high-quality Verilog code generation.
• ChatNeuroSim: An LLM Agent Framework for Automated Compute-in-Memory Accelerator Deployment and Optimization — ChatNeuroSim automates CIM accelerator design and optimization using LLM agents and design space pruning for faster, more efficient exploration.
• How LLMs Fail and Generalize in RTL Coding for Hardware Design? — LLMs struggle with parallel temporal logic in RTL coding, hitting a knowledge ceiling despite fine-tuning.
• SysVCoder: An LLM-Driven Framework for Systematic Generation of System-Level Design — A two-stage LLM framework using an intermediate representation significantly improves complex Verilog code generation.
• SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables — SPARTA automatically generates complex multi-hop Table-Text QA benchmarks, revealing weaknesses in current cross-modal reasoning models.
• not much happened today — AI agent development is shifting from interactive chat to robust, verifiable, and automated production systems.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.