Orthrus: toward evolutionary and functional RNA foundation models

2026-04-17 · Source: Machine learning : nature.com subject feeds · Field: Science & Research — Life Sciences & Biology, Mathematics & Computational Sciences, Research Methodology & Innovation · Depth: Expert, long

Summary

Orthrus is a novel Mamba-based mature RNA foundation model designed to predict key RNA properties and functions by leveraging biological domain knowledge. Unlike existing models that adapt textual domain strategies, Orthrus uses a self-supervised contrastive learning objective with biological augmentations. It maximizes embedding similarity between splice isoforms from ten model organisms and orthologous genes across over 400 mammalian species. This training approach creates latent representations that cluster RNA sequences based on functional and evolutionary similarities. Orthrus's mature RNA isoform representations demonstrate superior performance on mRNA property prediction tasks compared to other genomic foundation models, requiring significantly less fine-tuning data. The model also effectively captures the divergent biological functions of individual transcript isoforms, with its code and pretrained models publicly available on GitHub, Zenodo, and Hugging Face.

Key takeaway

For AI Scientists and Machine Learning Engineers developing genomic foundation models, Orthrus demonstrates that incorporating biological domain knowledge through contrastive learning significantly improves performance on RNA property prediction. You should consider adopting similar biologically-informed pretraining strategies to enhance model accuracy and reduce fine-tuning data requirements, especially when working with complex biological sequences like RNA.

Key insights

Orthrus is a Mamba-based RNA foundation model using contrastive learning and biological augmentations for superior RNA property prediction.

Principles

• Biological domain knowledge enhances RNA foundation models.
• Contrastive learning can cluster RNA by function and evolution.
• Mamba architecture is effective for RNA sequence modeling.

Method

Orthrus is pretrained using a self-supervised contrastive learning objective. It maximizes embedding similarity between splice isoforms from ten model organisms and orthologous genes from 400+ mammalian species, incorporating biological augmentations.

In practice

• Utilize Orthrus for mRNA property prediction tasks.
• Explore Orthrus embeddings for functional RNA clustering.
• Access pretrained models on Hugging Face for inference.

Topics

• RNA Foundation Models
• Mamba Architecture
• Contrastive Learning
• Transcript Isoform Function
• mRNA Property Prediction

Code references

• bowang-lab/Orthrus

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• Decoding the language of messenger RNA — The Orthrus RNA language model predicts mRNA properties by mapping evolutionary patterns across mammalian species.
• RNA design across eras: from covariance models to modern generative AI — RNA's functional versatility relies on 3D folding, with generative AI building on foundational probabilistic models for structure prediction.
• Fine-Tuning Biological Foundation Models with LoRA Using NVIDIA BioNeMo Recipes — LoRA enables efficient fine-tuning of large biological foundation models on single GPUs by adapting a minimal parameter subset.
• Programmable RNA translation through deep learning-driven IRES discovery and de novo generation — An AI framework unifies IRES identification, optimization, and de novo generation for programmable RNA translation.
• Rethinking Genomic Modeling Through Optical Character Recognition — OpticalDNA reframes genomic modeling as OCR-style document understanding, achieving efficiency and high-fidelity compression via visual layouts.
• A Four-Condition Diagnostic Protocol for Evidence Utilization in Long-Context and Retrieval-Augmented Language Models — A four-condition protocol diagnoses how long-context and RAG models utilize evidence, separating answer priors from true evidence-derived gains.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine learning : nature.com subject feeds.