Fine-Tuning Biological Foundation Models with LoRA Using NVIDIA BioNeMo Recipes

2026-06-15 · Source: NVIDIA Technical Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, medium

Summary

Low-Rank Adaptation (LoRA), integrated into NVIDIA BioNeMo Recipes, enables efficient fine-tuning of billion-parameter biological foundation models like ESM2 and Evo2. This approach makes adapting these large models practical on a single workstation GPU, such as the NVIDIA RTX 6000 Blackwell Workstation Edition, by training only a small fraction (~1-1.4%) of their parameters. Two case studies demonstrate this: fine-tuning ESM2-3B for protein secondary structure prediction (PSSP) and Evo2-1B for DNA splice-site classification. BioNeMo Recipes also incorporate performance optimizations like NVIDIA Transformer Engine (TE) and sequence packing, which improved throughput by approximately 5.5x for PSSP, ensuring competitive accuracy against strong published baselines while significantly reducing compute and memory requirements.

Key takeaway

For AI Engineers or Research Scientists adapting large biological foundation models, NVIDIA BioNeMo Recipes with LoRA offer a practical solution. You can achieve competitive accuracy for tasks like protein secondary structure prediction or DNA splice-site classification on a single NVIDIA RTX 6000 Blackwell Workstation Edition GPU. This approach significantly reduces compute and memory requirements, making billion-parameter model fine-tuning accessible and efficient for your specific biological applications.

Key insights

LoRA enables efficient fine-tuning of large biological foundation models on single GPUs by adapting a minimal parameter subset.

Principles

• LoRA freezes the backbone, trains low-rank adapter matrices.
• Sequence packing (THD) boosts throughput and memory efficiency.
• Parameter-efficient fine-tuning matches full fine-tuning quality.

Method

Fine-tune large biological foundation models by adding a task-specific head and LoRA adapters, keeping the backbone frozen, using tools like PEFT and BioNeMo Recipes.

In practice

• Use LoRA for ESM2-3B PSSP on RTX 6000.
• Apply LoRA for Evo2-1B DNA splice-site classification.
• Implement sequence packing for varying length datasets.

Topics

• Biological Foundation Models
• Low-Rank Adaptation
• NVIDIA BioNeMo Recipes
• Protein Secondary Structure Prediction
• DNA Splice-Site Classification
• Parameter-Efficient Fine-Tuning

Code references

• NVIDIA/bionemo-framework
• NVIDIA-NeMo/Megatron-Bridge
• NVIDIA/TransformerEngine
• huggingface/peft
• ArcInstitute/evo2

Best for: Machine Learning Engineer, Research Scientist, AI Engineer

Related on AIssential

• How to Fine-Tune a FLUX Model in under an hour with AI Toolkit and a DigitalOcean H100 GPU — Fine-tuning FLUX.1 LoRAs on custom data is achievable in under an hour using cloud GPUs and AI Toolkit.
• Carbon, open source DNA model, 250x faster than Evo2-7B and runs on llama.cpp — Applying LLM architectures to DNA sequences enables highly efficient genomic analysis and structure prediction.
• How to Accelerate Protein Structure Prediction at Proteome-Scale — Accelerated computing and optimized workflows enable proteome-scale prediction of protein complex structures.
• EnzymeTuning improves enzyme-constrained metabolic modeling and proteome abundance prediction through deep learning — EnzymeTuning, a GAN-based framework, optimizes *k*cat values to enhance metabolic modeling and proteome prediction accuracy.
• GRASS: Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning — GRASS adaptively samples LLM layers for fine-tuning based on gradient norms, significantly reducing memory while boosting performance.
• Scaling and quantization of a foundational deep learning model for network biology — Expanding Geneformer with more data and quantization improves biological prediction while reducing GPU demands.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Technical Blog.