A flaw in using pretrained protein language models in protein–protein interaction inference models

2026-02-13 · Source: Machine learning : nature.com subject feeds · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Computational Biology · Depth: Expert, medium

Summary

A study published in Nature Machine Intelligence on February 13, 2026, identifies and confirms a data leakage flaw in pretrained protein language models (pLMs) when applied to protein-protein interaction (PPI) inference tasks. The research characterizes this leakage by comparing pLMs trained on strict (leakage-controlled) and non-strict datasets, finding that existing pLMs inflate testing scores for PPI tasks. However, this inflation does not extend to non-paired biological tasks like protein keyword annotation. The study also found no correlation between pLM context lengths and performance on proteins exceeding those lengths. Furthermore, both pLM-based and non-pLM-based models struggled to generalize in predicting human-SARS-CoV-2 PPIs or the impact of point mutations on binding affinities, highlighting weaknesses in current pLM models and the need for improved evaluation protocols for paired biological datasets.

Key takeaway

For AI Researchers developing or applying pLMs for protein-protein interaction prediction, you must rigorously evaluate your models using datasets that strictly control for data leakage. Your evaluation protocols should extend beyond standard benchmarks to include out-of-distribution tasks, such as human-SARS-CoV-2 PPIs or mutation sensitivity, to accurately assess generalization capabilities and identify true model weaknesses.

Key insights

Pretrained protein language models exhibit data leakage in protein-protein interaction inference, inflating performance metrics.

Principles

• Data leakage inflates PPI task scores.
• Generalization remains a challenge for PPI models.

Method

The study characterized data leakage by training and comparing small, efficient pLMs on datasets with and without strict controls for leakage, assessing performance on PPI inference and other biological tasks.

In practice

• Use strict datasets for pLM training.
• Evaluate pLMs on out-of-distribution PPIs.

Topics

• Protein Language Models
• Protein-Protein Interaction
• Data Leakage
• Model Generalization
• Protein Function Prediction

Code references

• Emad-COMBINE-lab/pllm-ppi-data-leakage

Best for: AI Researcher, AI Scientist, Research Scientist

Related on AIssential

• Author Correction: Biophysical prediction of protein–peptide interactions and signaling networks using machine learning — Data quality is paramount for accurate machine learning models in biophysical predictions.
• Evaluating the role of pretraining dataset size and diversity on single-cell foundation model performance — Single-cell foundation models do not show clear data scaling laws, unlike large language models.
• A world model for proteins is here — Biohub's open-source protein "world model" (ESMFold2, ESM Atlas) accelerates drug discovery by mapping, predicting, and designing proteins.
• Knowledge Graphs and Reasoning LLMs for Finding Simple Yet Effective Transcriptomic Perturbation Predictors — Simple K-nearest neighbour models leveraging knowledge graphs, enhanced by RL-trained LLMs, offer competitive transcriptomic perturbation prediction.
• Viral Proteins Reveal Geometry of Protein Language Models — Protein language models organize sequence embeddings by a "nativeness axis" while preserving specific biological group signals.
• AWS and Hopkins Engineering announce groundbreaking database for AI/ML antibody design — A new diverse, experimentally validated dataset enables rigorous benchmarking of AI models for antibody developability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine learning : nature.com subject feeds.