On Sequence-to-Sequence Models for Automated Log Parsing

2026-06-12 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Data Science & Analytics · Depth: Expert, extended

Summary

This study systematically evaluated four sequence modeling architectures—Transformer, Mamba state-space, monodirectional LSTM, and bidirectional LSTM—for automated log parsing. Researchers trained 396 models across various dataset configurations, using relative Levenshtein edit distance for evaluation. Transformer achieved the lowest mean relative edit distance (0.111), followed by Mamba (0.145), mono-LSTM (0.186), and bi-LSTM (0.265). Mamba provided competitive accuracy with substantially lower computational cost. Character-level tokenization generally improved performance, while sequence length had negligible practical impact on Transformer accuracy. Both Mamba and Transformer demonstrated stronger sample efficiency than recurrent models, with Transformers reducing parsing error by 23.4%.

Key takeaway

For MLOps engineers designing log parsing pipelines, this study clarifies architectural choices. If your environment has structured or moderately variable log datasets, Mamba offers strong performance with significantly lower computational costs, making it ideal for budget-constrained deployments. For highly heterogeneous or distribution-shifted log formats, Transformers provide maximum robustness, reducing parsing error by 23.4%, despite higher compute demands. Evaluate character-level tokenization for improved accuracy.

Key insights

Mamba and Transformer models offer superior log parsing accuracy and sample efficiency, with Mamba providing cost advantages.

Principles

• Character-level tokenization generally improves log parsing performance.
• Mamba models offer competitive accuracy with substantially lower computational cost.
• Transformers demonstrate stronger robustness under log format heterogeneity.

Method

A controlled empirical study compared Transformer, Mamba, mono-LSTM, and bi-LSTM architectures, training 396 models across varied datasets and evaluating with relative Levenshtein edit distance.

In practice

• Prioritize character-level tokenization for log parsing when architectural constraints permit.
• Consider Mamba for log parsing in resource-constrained environments.
• Utilize Transformers for highly diverse or distribution-shifted log formats.

Topics

• Automated Log Parsing
• Sequence-to-Sequence Models
• Transformer Architecture
• Mamba State-Space Models
• Recurrent Neural Networks
• Computational Efficiency
• Log Data Analysis

Code references

• facebookresearch/hydra

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Mamba Unboxed: The State Space Model That’s Quietly Replacing Attention — Mamba-3, a State Space Model, surpasses Transformer performance in language modeling and long sequence processing.
• MambaBack: Bridging Local Features and Global Contexts in Whole Slide Image Analysis — MambaBack combines Mamba and Gated CNNs with spatial sampling and chunking for efficient WSI analysis.
• From RNNs to Transformers: How Sequence Models Evolved — Sequence models evolved by overcoming sequential processing and fixed context bottlenecks through dynamic attention and parallel computation.
• SSMamba: A Self-Supervised Hybrid State Space Model for Pathological Image Classification — Task-specific architectural designs and in-domain SSL significantly enhance pathological image analysis performance over generic FMs.
• Multilingual Coreference Resolution via Cycle-Consistent Machine Translation — Coreference resolution in low-resource languages can be significantly improved by cycle-consistent machine translation data augmentation.
• ​​Mamba4 Explained: A Faster Alternative to Transformers for Sequential Modeling — Mamba4 uses selective state space models for linear-time sequence processing, outperforming Transformers on long sequences.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.