Structural Generalization on SLOG without Hand-Written Rules

2026-04-30 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

A novel neural cellular automaton (NCA) with a discrete bottleneck has been developed to address structural generalization in semantic parsing, specifically on the SLOG benchmark. This system learns compositional rules from data through local iteration, eliminating the need for hand-written algebraic rules typically used by models like AM-Parser. The NCA achieved a 100% type-exact match on 11 of 17 structural generalization categories, outperforming AM-Parser in three categories where it scored between 0% and 74%. The system demonstrated high consistency with an overall standard deviation of 0.2 across 10 seeds, significantly lower than AM-Parser's 4.3. Analysis revealed that all 5,539 failure instances were attributable to two specific mechanisms: novel combinations of wh-extraction context with reduced verb types, and modifiers on the subject side of verbs.

Key takeaway

For research scientists developing semantic parsers, this work demonstrates a viable alternative to rule-based systems for structural generalization. You should consider neural cellular automata with discrete bottlenecks to learn compositional rules directly from data, potentially reducing manual engineering effort and improving generalization performance on novel structural combinations.

Key insights

A neural cellular automaton can achieve structural generalization in semantic parsing without hand-written rules.

Principles

• Local iteration enables learning compositional rules.
• Discrete bottlenecks can facilitate structural generalization.

Method

The system employs a neural cellular automaton (NCA) with a discrete bottleneck to learn compositional rules from data via local iteration, avoiding explicit rule engineering.

In practice

• Apply NCA for structural generalization tasks.
• Analyze failure modes for specific structural patterns.

Topics

• Structural Generalization
• Semantic Parsing
• Neural Cellular Automaton
• Discrete Bottleneck
• SLOG Benchmark

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Dependency Parsing Across the Resource Spectrum: Evaluating Architectures on High and Low-Resource Languages — Biaffine LSTMs excel in low-resource dependency parsing, outperforming Transformers until sufficient data is available.
• (1D) Ordered Tokens Enable Efficient Test-Time Search — Coarse-to-fine tokenization improves test-time search steerability in autoregressive generative models.
• Geometry of Semantic Space: Comparative Study of Discrete and Continuous Models — Graph-based semantic models offer more interpretable and structured meaning representations than high-dimensional transformer embeddings.
• Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional data — Self-supervised local learning rules can effectively uncover hierarchical data structures, aligning with biological plasticity.
• Learning to Theorize the World from Observation — Understanding the world means inferring explicit, compositional programs from raw observations, not just predicting future states.
• Towards Generalization of Block Attention via Automatic Segmentation and Block Distillation — Semantic segmentation and an efficient distillation framework enable practical, scalable block attention for long-context LLMs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.