Discovery of candidate therapeutic targets with Geneformer

2026-04-23 · Source: Machine learning : nature.com subject feeds · Field: Science & Research — Life Sciences & Biology, Health & Medical Research · Depth: Intermediate, medium

Summary

Geneformer is a foundational artificial intelligence model designed to identify gene networks disrupted in disease, particularly in data-limited settings. Pretrained on over 100 million single-cell transcriptomes, it enables context-aware predictions in network biology. The methodology involves tokenizing raw gene expression counts into rank value encodings, assessing phenotype separability with zero-shot embeddings, and then fine-tuning. Fine-tuning can be single-task, such as disease prediction within a specific cell type, or multi-task to learn cross-informative features like cell types and disease states. Performance is evaluated using confusion matrices, macro F1 scores, and embedding analysis. The protocol also supports in silico perturbation to simulate gene repression or activation, quantifying cell state shifts to prioritize therapeutic targets. This process, including an option for quantized models for efficiency, typically completes in under 2 days on a standard GPU workstation and requires moderate Python experience.

Key takeaway

For AI Scientists and Research Scientists working on therapeutic target discovery with limited single-cell data, Geneformer offers a robust, efficient solution. Its ability to perform context-aware predictions and in silico perturbations can accelerate the identification of disease-disrupting gene networks. You should explore its fine-tuning capabilities for specific disease contexts and leverage its computational efficiency for rapid prototyping.

Key insights

Geneformer leverages a transformer model and transfer learning for gene network analysis and therapeutic target discovery with limited data.

Principles

• Pretraining on large datasets enables context-aware predictions.
• Zero-shot inference assesses phenotype separability.
• Quantized models enhance computational efficiency.

Method

The Geneformer protocol involves tokenization of gene expression, zero-shot embedding assessment, fine-tuning (single or multi-task), and in silico perturbation to identify therapeutic targets.

In practice

• Use Geneformer for disease prediction in specific cell types.
• Apply in silico perturbation to prioritize candidate gene targets.
• Utilize quantized models for faster analysis on GPUs.

Topics

• Geneformer
• Single-cell Transcriptomics
• Therapeutic Target Discovery
• In Silico Perturbation
• Foundation Models

Best for: AI Scientist, Research Scientist

Related on AIssential

• OCOO-T : A Simple and Scalable Virtual Cell Model for Transcriptional Perturbation Response Prediction — OCOO-T uses a simple Transformer and flow matching to directly denoise gene expression for perturbation prediction.
• scLong: a billion-parameter foundation model for capturing long-range gene context in single-cell transcriptomics — scLong is a billion-parameter foundation model for scRNA-seq, capturing long-range gene dependencies and integrating external gene knowledge.
• Single-round evolution of RNA aptamers with GRAPE-LM — GRAPE-LM enables single-round RNA aptamer evolution using AI, outperforming multi-round human methods.
• A new tool is revealing the invisible networks inside cancer — RNACOREX reveals hidden gene networks in cancer, predicting survival with AI-level accuracy and interpretable molecular explanations.
• $\textit{BlockFormer}$ : Transformer-based inference from interaction maps — BlockFormer infers per-entity parameters from variable interaction maps using synthetic data and a block-aware transformer with 3D positional encoding.
• Predicting and interpreting cell-type-specific drug responses in the small-data regime using inductive priors — PrePR-CT uses graph-based deep learning with cell-type-specific co-expression networks to predict drug responses in data-limited settings.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine learning : nature.com subject feeds.