Grammar of the Wave: Towards Explainable Multivariate Time Series Event Detection via Neuro-Symbolic VLM Agents

2026-06-12 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

"Grammar of the Wave" introduces Knowledge-Guided Time Series Event Detection (K-TSED), a new task addressing the challenge of identifying semantically complex events in multivariate signals with limited labeled data. The authors propose Event Logic Tree (ELT), a novel knowledge representation framework that translates natural language event descriptions into hierarchical temporal-logic structures. This framework underpins SELA, a neuro-symbolic VLM agent system that iteratively instantiates signal primitives and composes them according to ELT constraints, providing both event detection and faithful explanations. To validate this approach, a new benchmark, KITE, was created using 41 real-world energy production time series datasets with expert annotations. Experiments demonstrate SELA's superior performance over supervised fine-tuning baselines and existing zero-shot LLM/VLM methods like Numeric and VL-Time, achieving F1@0.5 scores close to human experts and effectively mitigating VLM hallucination.

Key takeaway

For AI Scientists and Machine Learning Engineers developing explainable time series event detection in low-resource, high-stakes domains, you should explore neuro-symbolic VLM agent frameworks like SELA. This approach, leveraging Event Logic Trees, enables robust zero-shot detection from natural language descriptions, significantly improving performance and providing verifiable explanations. It also effectively mitigates VLM hallucination, fostering greater trust in automated event identification.

Key insights

Neuro-symbolic VLM agents utilize Event Logic Trees for explainable, zero-shot time series event detection from natural language descriptions.

Principles

• Events require hierarchical, semantically quantified, and temporally elastic representation.
• A physical channel supports only one active primitive state at any time point.
• Temporal conjunctions (SEQ, SYNC, GUARD) and disjunction (OR) form a complete operator basis.

Method

SELA uses Logic Analyst agents to build ELT schemas from text and Signal Inspector agents to instantiate them on time series data via active visualization tools.

In practice

• Utilize ELT to translate natural language event descriptions into structured temporal logic.
• Employ active visualization tools to refine time series interval detection.
• Benchmark K-TSED solutions using the KITE dataset from energy production.

Topics

• Knowledge-Guided TSED
• Neuro-Symbolic AI
• Vision-Language Models
• Event Logic Tree
• Multi-Agent Systems
• Explainable AI
• KITE Dataset

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Harnessing Generalist Agents for Contextualized Time Series — TimeClaw's agentic harness framework enables LLMs to perform robust, contextualized time series reasoning by providing native temporal runtime support.
• EXCODER: EXplainable Classification Of DiscretE time series Representations — Discrete latent representations enhance time series classification explainability by reducing data redundancy.
• Data Machina #261 — Generative AI and LLMs are being adapted to achieve state-of-the-art performance in diverse time-series tasks.
• E4GEN: Event-level Explainable Extreme-Enhanced Time-series Generation — E4GEN is an explainable diffusion framework that precisely controls extreme event generation in time series without disrupting overall fidelity.
• Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities — Neuro-Symbolic AI combines neural pattern recognition with symbolic reasoning to enhance cybersecurity defense and offense.
• A modular architecture for creating multimodal embodied agents with an episodic Knowledge Graph as an explainable and controllable long-term memory — A modular platform uses an event bus and episodic Knowledge Graph for controllable, explainable multimodal embodied agents.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.