Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies

2026-07-11 · Source: cs.AI updates on arXiv.org · Field: Health & Wellbeing — Medical Devices & Health Technology, Artificial Intelligence & Machine Learning, Clinical Care & Medical Practice · Depth: Expert, extended

Summary

This study evaluates sequence-based neural networks and data augmentation strategies for automated detection of autism-related self-stimulatory behaviors from video, addressing the challenge of limited scalable computational methods for remote behavioral screening. Using the Self-Stimulatory Behavior Diagnosis (SSBD) dataset, LSTM and GRU models were trained on pose-derived features. Both architectures surpassed prior CNN baselines (62–76% accuracy), achieving peak accuracies of 97.5% (LSTM) and 98.75% (GRU) at a sampling interval of every 15 frames. Additionally, ten data augmentation strategies were applied within an I3D transfer learning pipeline. Horizontal flip yielded the highest standalone accuracy (48.78%), while upsampling was identified as the most critical component, with its exclusion causing the largest performance degradation (loss 5.0101). A personalized machine learning approach also showed consistent predictions (mean loss 1.84, SD 0.79).

Key takeaway

For Machine Learning Engineers developing video-based autism screening tools, prioritize GRU architectures and a 15-frame sampling interval for optimal accuracy and computational efficiency. When working with small behavioral datasets, integrate upsampling as a critical data augmentation strategy to mitigate data scarcity. Consider personalized models for within-video generalization, but validate minimum calibration duration for reliable performance.

Key insights

Sequence-based models with optimal temporal sampling significantly outperform CNNs for autism-related behavioral classification.

Principles

• Explicit temporal modeling improves behavioral classification.
• Optimal sampling balances dynamics and noise.
• Upsampling is critical for small video datasets.

Method

LSTM and GRU models were trained on pose features from the SSBD dataset at various frame sampling intervals. An I3D transfer learning pipeline was used with ten data augmentation strategies, including an ablation study.

In practice

• Use GRU for high accuracy and efficiency.
• Sample video features every 15 frames.
• Prioritize upsampling in augmentation pipelines.

Topics

• Autism Spectrum Disorder
• Behavioral Classification
• Recurrent Neural Networks
• Temporal Sampling
• Data Augmentation
• I3D Transfer Learning

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies — Optimal frame rates and specific data augmentation significantly boost video-based autism behavior classification accuracy.
• Spatial-Temporal Decoupled Adapter for Micro-gesture Online Recognition — A Spatial-Temporal Decoupled Adapter combined with Adaptive Soft Balanced Augmentation significantly enhances micro-gesture online recognition performance.
• Swift Sampling: Selecting Temporal Surprises via Taylor Series — Swift Sampling identifies critical video moments by detecting deviations from predicted visual feature evolution using Taylor series.
• A machine learning study highlighting the challenges of fidgety movement recognition using vision and inertial sensors — Automated Fidgety Movement recognition in infants faces significant generalization challenges across unseen subjects.
• Dynamic Cluster Data Sampling for Efficient and Long-Tail-Aware Vision-Language Pre-training — Dynamic cluster-based sampling efficiently trains VLMs, improving long-tail concept performance by balancing data distribution.
• Exploring Data Augmentation and Resampling Strategies for Transformer-Based Models to Address Class Imbalance in AI Scoring of Scientific Explanations in NGSS Classroom — Targeted data augmentation significantly improves transformer-based model performance in automated scoring of imbalanced educational text data.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.