Stylistic-STORM (ST-STORM) : Perceiving the Semantic Nature of Appearance

2026-04-20 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

ST-STORM is a novel self-supervised learning (SSL) framework designed to disentangle visual appearance (style) from content, addressing limitations of traditional SSL methods that often discard appearance cues critical for fine-grained tasks. The architecture features two distinct latent streams: a Content branch, which uses a JEPA scheme with a contrastive objective to achieve stable semantic representations invariant to appearance, and a Style branch, which captures appearance signatures (textures, contrasts, scattering) via Style-JEPA and adversarial reconstruction. ST-STORM leverages style transfer and spectral perturbations to create "stylistic chaos" for controlled appearance alteration. Evaluated on ImageNet-1K for object classification, fine-grained weather characterization (Multi-Weather dataset), and melanoma detection (ISIC 2024 Challenge), the Style branch achieved F1 scores of 97% on Multi-Weather and 94% on ISIC 2024 with 10% labeled data, without degrading the Content branch's semantic performance (F1=80% on ImageNet-1K). The framework improves critical appearance information preservation compared to MoCo-v3 and I-JEPA.

Key takeaway

For research scientists developing self-supervised learning models for critical applications like autonomous driving or medical diagnostics, ST-STORM offers a robust approach. You should consider implementing its dual-branch architecture to explicitly disentangle content and style. This allows for preserving crucial appearance information, which traditional invariance-based methods often discard, thereby enhancing performance on fine-grained tasks without compromising general semantic understanding.

Key insights

ST-STORM disentangles image content and style into separate, predictable latent spaces for robust, fine-grained visual analysis.

Principles

• Appearance can be a semantic modality.
• Predictability filters contingent details.
• Invariance is not always beneficial.

Method

ST-STORM uses a U-Net for content and a pyramidal encoder for style, fused via SPADE blocks. It employs Style-JEPA for predictable style tokens and MoCo-style contrastive learning for content invariance.

In practice

• Use ST-STORM for fine-grained weather analysis.
• Apply ST-STORM to improve melanoma detection.
• Leverage style tokens for appearance-critical tasks.

Topics

• Self-supervised Learning
• Stylistic-STORM
• Appearance Disentanglement
• Joint-Embedding Predictive Architecture
• Contrastive Learning

Code references

• Hamedkiri/Weather_MultiTask_Datasets
• Hamedkiri/RT-STORM-V2

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Stylistic-STORM (ST-STORM) : Perceiving the Semantic Nature of Appearance — Appearance can be a semantic signal, not just noise, requiring dedicated disentanglement in self-supervised learning.
• Rank-Turbulence Delta and Interpretable Approaches to Stylometric Delta Metrics — New Delta metrics offer interpretable, high-accuracy authorship attribution by treating word frequencies as probability distributions.
• Show, Don't Ask: Generative Visual Disambiguation for Composed Image Retrieval with Turn-Valid Coverage — CLARA uses visual prototypes and reweighted conformal prediction to resolve CIR ambiguity more effectively than text questions.
• FreeStyle: Free Control of Style-Content Dual-Reference Generation from Community LoRA Mining — FreeStyle uses community LoRAs and a two-stage curriculum for robust style-content dual-reference image generation.
• Semantic Smoothing via Novel View Synthesis for Robust SAR Image Classification — Semantic smoothing uses generative models to create semantically consistent views for robust classification, outperforming isotropic noise.
• Texture-Shape Bias Balancing for Robust Synthetic-to-Real Semantic Segmentation in Automotive NIR Imagery — A generative augmentation framework reduces the synthetic-to-real domain gap for automotive NIR semantic segmentation by balancing texture-shape bias.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.