Agentic Neural Architecture Search

2026-04-28 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Automated Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

AgentNAS introduces a novel mechanism that integrates large language model (LLM) driven design with conventional Neural Architecture Search (NAS). This approach addresses the limitation of manually engineered search spaces by having an LLM generate a high-quality seed architecture, which is then decomposed into a "slotted architecture." This scaffold, featuring named, interchangeable module slots, automatically defines a bounded, task-specific search space for NAS to explore without manual intervention. Implemented as a modular three-phase pipeline, AgentNAS achieved new leading performance on 11 of 17 diverse tasks across NAS-Bench-360 and Unseen NAS benchmarks, covering classification, dense regression, segmentation, and multi-label tagging. Ablation studies confirm the LLM's initial seed often outperforms baselines, with NAS providing further combinatorial gains. The method's robustness was demonstrated across Claude Sonnet 4.6, Haiku 4.5, and Opus 4.6.

Key takeaway

For AI Architects and Machine Learning Engineers struggling with the manual burden of designing neural network search spaces, you should explore integrating LLM-driven architecture generation with conventional NAS. This approach, exemplified by AgentNAS, allows an LLM to create a strong initial architecture and a tailored search space, which NAS then refines for superior performance on diverse tasks. Consider this hybrid strategy to automate architecture discovery, but be mindful that LLM-generated spaces may not always surpass highly optimized hand-designed ones for every specific task.

Key insights

LLMs can construct task-specific, bounded search spaces for conventional Neural Architecture Search to refine LLM-generated seed architectures.

Principles

• LLMs expand architecture space; NAS explores bounded spaces combinatorially.
• Slotted architectures preserve macro-structure while exposing module-level freedom.
• LLM-designed search spaces can outperform grammar-based alternatives.

Method

AgentNAS follows a three-phase pipeline: LLM-driven seed search, slotted architecture synthesis, and conventional NAS exploration.

In practice

• Generate initial high-quality network seeds using LLMs.
• Decompose architectures into interchangeable module slots for NAS.
• Include Identity/Zero alternatives in slots for NAS-driven pruning.

Topics

• Neural Architecture Search
• Large Language Models
• LLM Agents
• Automated Machine Learning
• Slotted Architectures
• Model Design Automation

Code references

• alroimfebruary/AgentNAS

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Architect

Related on AIssential

• Agentic Neural Architecture Search — AgentNAS combines LLMs and conventional NAS to automate architecture search space definition, achieving leading performance across diverse tasks.
• Algorithmic algorithm development with LLMs: A Case Study on LLM-Usage for Contraction Order Optimization in Tensor Networks — LLMs can develop and improve algorithms for complex problems like tensor network optimization, guided by verifiers, but human oversight remains crucial.
• Output-Space Search: Targeting LLM Generations in a Frozen Encoder-Defined Output Space — OS-Search reframes LLM generation as endpoint search in a latent space, enabling parallel optimization and enhanced diversity.
• Agentic Discovery of Neural Architectures: AIRA-Compose and AIRA-Design — LLM agents can autonomously discover and optimize neural architectures and training methods, surpassing human-designed baselines.
• An Agentic AI Scientific Community for Automated Neural Operator Discovery — An agentic AI community automates neural operator discovery, demonstrating LLM-driven hybridization and a "no-free-lunch" principle.
• ChatNeuroSim: An LLM Agent Framework for Automated Compute-in-Memory Accelerator Deployment and Optimization — ChatNeuroSim automates CIM accelerator design and optimization using LLM agents and design space pruning for faster, more efficient exploration.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.