LLMCFG-TGen: Using LLM-Generated Control Flow Graphs to Automatically Create Test Cases from Use Cases

2026-06-26 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, extended

Summary

LLMCFG-TGen is an end-to-end approach designed to automatically generate comprehensive and accurate test cases directly from Natural Language (NL) use-case descriptions. This method addresses limitations in current LLM-based test generation, which often struggle with coverage and complex conditional logic. LLMCFG-TGen operates in three steps: an LLM first transforms a use case into a structured, JSON-based Control Flow Graph (CFG), which is then explored to extract all complete execution paths. Finally, these paths are used to create detailed test cases. Evaluated across four multi-domain datasets comprising 42 use cases, the system demonstrated high accuracy in CFG generation, achieving an average node-level F1 score of 0.895 and an nGED of 0.933 compared to manually constructed CFGs. It achieved full path coverage with a 2.38% discrepancy rate and an average absolute difference of 0.02 in test case counts, outperforming baseline methods. Practitioner feedback confirmed the logical consistency, comprehensiveness, and significant reduction in manual effort.

Key takeaway

For QA managers and test engineers seeking to automate test case generation from natural language use cases, LLMCFG-TGen offers a robust solution. By leveraging LLM-generated Control Flow Graphs, this approach ensures comprehensive path coverage and significantly reduces manual effort compared to traditional or direct LLM methods. You should consider integrating this CFG-guided pipeline to produce logically consistent, non-redundant test suites, improving overall software quality and accelerating your testing cycles.

Key insights

Coupling LLM semantic reasoning with structured Control Flow Graph modeling effectively bridges NL requirements to systematic test generation.

Principles

• LLMs can effectively construct structured CFGs from NL use cases.
• CFG-guided generation enhances test coverage and reduces redundancy.
• Automated validation of LLM-generated CFGs ensures structural completeness.

Method

An LLM generates a JSON-based CFG from NL use cases, which is then traversed via DFS to extract execution paths. These paths are finally converted into detailed test cases by an LLM.

In practice

• Automate test case generation from use cases using LLM-generated CFGs.
• Implement CFG validation to ensure structural integrity and completeness.
• Derive test cases from execution paths for comprehensive coverage.

Topics

• Large Language Models
• Test Case Generation
• Control Flow Graphs
• Requirements-Based Testing
• Software Testing Automation
• Use Case Analysis

Code references

• aimeenorwind/LLMCFG-TGen

Best for: AI Engineer, NLP Engineer, Research Scientist, AI Scientist, Software Engineer, Machine Learning Engineer

Related on AIssential

• Knowledge Matters: Injecting Project and Testing Knowledge into LLM-based Unit Test Generation — KTester enhances LLM-based unit test generation by integrating project-specific and testing domain knowledge through a modular, multi-perspective pipeline.
• AI-Driven Test Case Generation from Natural Language Requirements: A Survey of Techniques and Research Gaps — Automating test case generation from natural language requirements with AI faces significant challenges, particularly in quality dimensions like hallucination and traceability.
• Library-Aware Doubles and Iterative Repair for Large Language Model-Generated Unit Tests in OpenSIL Firmware — LLM-guided, multi-agent pipelines can automate unit test generation and repair for constrained firmware environments.
• Meta Applies Mutation Testing with LLM to Improve Compliance Coverage — LLMs can significantly improve mutation testing by generating context-aware mutants and targeted tests, enhancing compliance coverage.
• CAF-Gen: A Multi-Agent System for Enriching Argumentation Structures — Multi-agent LLM systems with iterative Creator-Reviewer pipelines reliably enrich shallow argument structures into complex formal models.
• FlowPipe: LLM-Enhanced Conditional Generative Flow Networks for Data Preparation Pipeline Construction — FlowPipe uses LLM-enhanced C-GFlowNets to efficiently construct data preparation pipelines, improving accuracy and training speed.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.