XRFormer: Multiscale Tokenization for XRF Representation Learning

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

XRFormer is a novel transformer architecture specifically designed for analyzing complex one-dimensional X-ray fluorescence (XRF) spectra, crucial for material analysis in cultural heritage. It features a multiscale convolutional tokenizer that injects locality and multi-resolution inductive biases before global self-attention, progressively reducing spectral resolution while increasing embedding dimensionality. The model also incorporates self-supervised pretraining using Masked Spectral Modeling (MSM) and a physics-informed Peak Presence Prediction (PPP) objective. Experiments on the Pigments Checker STANDARD v.5 dataset demonstrate XRFormer's superior performance, consistently outperforming ViT, SpectralFormer (without CAF), and a 1D-CNN baseline for pigment identification. For pigment unmixing, XRFormer achieves robust abundance estimation with significantly higher parameter efficiency, using 1.5M parameters and 128 tokens compared to SpectralFormer's 3.37M parameters and 512 tokens. MSM and PPP pretraining further enhance performance, with the MSM+PPP variant achieving 76.78% Absolute Accuracy for identification and 0.0399 A-RMSE for unmixing.

Key takeaway

For AI Scientists and Machine Learning Engineers developing models for XRF analysis in cultural heritage, you should prioritize modality-aware tokenization. XRFormer's multiscale convolutional tokenizer and self-supervised pretraining, especially with Masked Spectral Modeling and Peak Presence Prediction, offer superior performance and parameter efficiency. Consider adopting these techniques to enhance pigment identification and unmixing accuracy, particularly when working with data-limited spectral datasets.

Key insights

Multiscale, modality-aware tokenization significantly improves transformer performance for complex 1D spectral data like XRF.

Principles

• Combining locality with multi-resolution analysis enhances spectral feature capture.
• Self-supervised pretraining improves representation quality with limited labeled data.
• Physics-informed pretext tasks can further align models with domain characteristics.

Method

XRFormer uses a multiscale convolutional tokenizer to reduce spectral resolution and increase embedding dimensionality, followed by a standard transformer encoder. Self-supervised pretraining involves Masked Spectral Modeling and Peak Presence Prediction.

In practice

• Apply multiscale convolutional tokenizers to 1D spectral data for improved feature extraction.
• Implement Masked Spectral Modeling for general spectral structure learning.
• Use Peak Presence Prediction to inject physics-informed biases for XRF analysis.

Topics

• X-ray Fluorescence
• Transformer Models
• Multiscale Tokenization
• Self-Supervised Learning
• Cultural Heritage
• Pigment Identification
• Spectral Unmixing

Code references

• sofiane1010/XRFormer

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Spectra: Rethinking Optimizers for LLMs Under Spectral Anisotropy — LLM gradients exhibit a "spike-tail" anisotropy where dominant linguistic structures suppress long-tail semantic learning.
• X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation — X-Token enables cross-tokenizer knowledge distillation by projecting student distributions into teacher vocabulary space, overcoming prior matching failures.
• RePercENT: Scaling Disentangled Representation Learning Beyond Two Modalities — RePercENT scales disentangled representation learning beyond two modalities using a self-supervised, plug-and-play framework.
• Kronecker Embeddings: Byte-Level Structured Token Representations for Parameter-Efficient Language Models — Kronecker Embeddings drastically cut LLM input-side parameters while improving performance and spelling robustness.
• Beyond N-gram: Data-Aware X-GRAM Extraction for Efficient Embedding Parameter Scaling — X-GRAM improves embedding efficiency by dynamically managing token frequency and integrating refined n-gram features.
• Unmixing ATR-{\mu}FTIR spectroscopic images of cross-sections of historical oil paintings — An unsupervised CNN autoencoder with a weighted spectral angle distance loss effectively unmixes complex ATR-μFTIR data from historical paintings.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.