$\textit{BlockFormer}$ : Transformer-based inference from interaction maps

2026-05-23 · Source: cs.LG updates on arXiv.org · Field: Science & Research — Life Sciences & Biology, Mathematics & Computational Sciences, Research Methodology & Innovation · Depth: Expert, extended

Summary

BlockFormer is a novel transformer-based model designed for inference from interaction maps, such as identifying centromeres from genome-wide Hi-C data. This data-driven approach effectively handles variability in the number and size of interacting entities by employing a block-aware transformer architecture with a three-dimensional positional encoding. A custom simulator generates abundant, computationally cheap synthetic data for training. Applied to centromere localization, BlockFormer accurately recovers genomic positions across diverse species, achieving near-resolution accuracy (e.g., 0.58 for S.M., 1.22 for L.K.) and sub-resolution precision (e.g., 0.08 error or ~2kb for S. cerevisiae at 32kb resolution). It consistently outperforms the non-amortized Centurion method in speed and often accuracy, for instance, 0.27 error in 24.6 seconds versus 185.5 seconds for P.F.s. The model, trained for 15 hours on an NVIDIA TITAN X (Pascal) GPU, also demonstrates robustness to varying block sizes, numbers, sequencing depths, and diverse spot patterns, and can be adapted for tasks like loop localization.

Key takeaway

For Research Scientists or Machine Learning Engineers analyzing genomic interaction maps, BlockFormer provides a robust, amortized solution for tasks like centromere or loop localization. You should consider integrating this transformer-based approach to achieve sub-resolution accuracy and significantly faster inference times compared to non-amortized methods like Centurion. Its flexibility to varying block sizes, numbers, and sequencing depths makes it highly adaptable across diverse species and experimental conditions, offering a powerful tool for efficient and precise genomic feature identification.

Key insights

BlockFormer infers per-entity parameters from variable interaction maps using synthetic data and a block-aware transformer with 3D positional encoding.

Principles

• Interaction maps encode system structure for entity-level inference.
• Simulated data can bypass costly manual annotation for model training.
• Block-aware architectures are crucial for variable interaction map structures.

Method

BlockFormer decomposes inference into per-entity subproblems, uses per-block patching and 3D positional encoding, and trains on diverse synthetic maps to generalize across block numbers and sizes.

In practice

• Employ 3D positional encoding for transformer models on block-wise data.
• Generate simplified synthetic data to train models for complex biological maps.
• Apply iterative refinement for out-of-training range chromosome sizes.

Topics

• BlockFormer
• Transformer Architecture
• Hi-C Maps
• Centromere Localization
• Genomic Analysis
• Simulation-Based Inference

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• Identifying spatial single-cell-level interactions with graph transformer — A self-supervised graph transformer identifies spatial single-cell interactions without known ligand-receptor pairs.
• $p$-adic Bi-Filtrations for Topological Machine Learning on Genomic Sequences — pVR combines p-adic numbers and TDA for alignment-free genomic sequence classification, particularly effective in low-sample settings.
• Discovery of candidate therapeutic targets with Geneformer — Geneformer leverages a transformer model and transfer learning for gene network analysis and therapeutic target discovery with limited data.
• Prediction and functional interpretation of inter-chromosomal genome architecture from DNA sequence with TwinC — TwinC predicts inter-chromosomal DNA contacts, revealing their mechanistic drivers and role in gene regulation.
• OCOO-T : A Simple and Scalable Virtual Cell Model for Transcriptional Perturbation Response Prediction — OCOO-T uses a simple Transformer and flow matching to directly denoise gene expression for perturbation prediction.
• scGTN: Deep Siamese Graph Transformer Network for Single-cell RNA Sequencing Clustering — scGTN integrates gene expression and intercellular structure via a Siamese graph transformer for robust scRNA-seq clustering.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.