The test suite as a regression sensor

2026-05-27 · Source: Martin Fowler · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, extended

Summary

This article details experiments with "sensors" designed to enhance the maintainability of codebases generated by coding agents, specifically for an internal TypeScript, NextJS, and React analytics dashboard. The author used a mix of Claude Sonnet, Claude Opus, and Cursor's composer-2 models. Experiments covered static code analysis, including ESLint with custom rules for AI shortcomings (e.g., max arguments, file/function length, cyclomatic complexity), and dependency-cruiser for enforcing module layering. Custom coupling metrics were explored, but raw data proved less useful than AI-driven modularity reviews using Vlad Khononov's "Modularity Skills," which identified issues like duplicate route code and inconsistent backend calls. Mutation testing with Stryker was found crucial for detecting assertion gaps in AI-generated test suites, even with 100% statement coverage, highlighting its role as a regression sensor. The article concludes that computational sensors excel at file-level issues, while inferential (LLM-based) sensors are vital for semantic interpretation of cross-file concerns like modularity.

Key takeaway

For AI Engineers building with coding agents, you must integrate a robust sensor system to manage codebase maintainability and technical debt. Relying solely on AI-generated tests or basic linting creates a false sense of security; instead, implement custom static analysis rules and crucial mutation testing to verify test effectiveness. Your review process should also incorporate LLM-based modularity reviews, as they provide semantic interpretation beyond raw metrics, helping you identify and address complex design issues early.

Key insights

AI-generated code maintainability benefits from a system of computational and inferential sensors.

Principles

• Custom sensor feedback guides AI self-correction.
• LLMs provide semantic interpretation for complex code issues.
• Mutation testing reveals true test suite effectiveness.

Method

Implement a sensor system combining computational tools (ESLint, dependency-cruiser, type checkers, mutation testing) with inferential LLM-based reviews for maintainability feedback to coding agents.

In practice

• Configure custom ESLint formatters for AI guidance.
• Use dependency-cruiser to enforce module layering.
• Employ mutation testing to validate AI-generated tests.

Topics

• Coding Agents
• Code Maintainability
• Static Code Analysis
• Dependency-cruiser
• Mutation Testing
• LLM-based Code Review

Code references

• Factory-AI/eslint-plugin
• sverweij/dependency-cruiser
• vladikk/modularity

Best for: AI Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• AI tools for hearing difficulties — helpful or harmful for language learning? — AI assistive technology presents a dilemma for language learners with hearing difficulties, balancing aid with skill development.
• How Enterprise Marketing Teams Are Actually Using AI Agents in 2026 — Enterprise marketing is shifting from human-assisted AI tools to autonomous AI agents that redesign entire workflows.
• [AINews] Microsoft Build: MAI-Thinking-1 and MAI Family models — Microsoft's MAI models showcase a transparent, full-stack AI strategy emphasizing clean data and local agent execution.
• Are we getting better at explaining things because of our interaction with LLMs? — Crafting clear LLM prompts may enhance human explanatory and communication abilities.
• OpenAI expands Codex with role-specific plugins to build a general-purpose app for non-developers — OpenAI is transforming Codex into a general-purpose, role-specific AI work app, expanding beyond coding for non-developers.
• How to Use Claude Managed Agents? — Anthropic's Claude Managed Agents simplifies AI agent deployment by providing a hosted, stateful, and secure execution environment.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Martin Fowler.