Bolek: A Multimodal Language Model for Molecular Reasoning

2026-05-04 · Source: Machine Learning · Field: Science & Research — Artificial Intelligence & Machine Learning, Life Sciences & Biology, Health & Medical Research · Depth: Expert, quick

Summary

Bolek is a compact multimodal language model designed for molecular reasoning, addressing the auditability challenges of existing drug-discovery models. It integrates a Morgan fingerprint embedding into an instruction-tuned text decoder, grounding natural-language reasoning in molecular structure. Fine-tuned on molecular alignment tasks like description, RDKit descriptor prediction, and substructure detection, Bolek also performs downstream reasoning on 15 TDC binary classification tasks using synthetic chains-of-thought. The model significantly outperforms its Qwen3-4B-Instruct base, increasing mean ROC/PR AUC from 0.55 to 0.76, and surpasses TxGemma-9B-Chat on most classification tasks despite being smaller. Bolek's explanations are more grounded, citing numerical descriptors 10-100 times more frequently with strong agreement to RDKit values for key descriptors.

Key takeaway

For AI engineers developing drug-discovery models, Bolek demonstrates that integrating molecular structure embeddings directly into language models can yield more auditable and accurate reasoning. You should consider multimodal input and synthetic, feature-anchored chains-of-thought to improve model performance and the verifiability of generated explanations, especially when working with compact models for high-stakes applications.

Key insights

Targeted multimodal injection and reasoning supervision create auditable, compact molecular reasoning models.

Principles

• Ground reasoning in concrete features.
• Multimodal input enhances model auditability.
• Synthetic chains-of-thought improve performance.

Method

Bolek injects Morgan fingerprint embeddings into a Qwen3-4B-Instruct text decoder, then fine-tunes on molecular alignment and 15 TDC binary classification tasks using synthetic chains-of-thought anchored in molecular features.

In practice

• Use Morgan fingerprints for molecular grounding.
• Generate synthetic rationales for training.
• Integrate RDKit descriptors for validation.

Topics

• Bolek
• Multimodal Language Models
• Molecular Reasoning
• Morgan Fingerprint
• Drug Discovery

Best for: AI Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Multi-to-uni modal knowledge transfer pre-training for molecular representation learning — M2UMol enables robust molecular representation learning from incomplete multimodal data by transferring knowledge to a 2D encoder.
• Augmenting Molecular Language Models with Local $n$-gram Memory — Integrating local $n$-gram memory via MolGram efficiently resolves the locality gap in molecular language models, improving performance with fewer parameters.
• Bridging three-dimensional molecular structures and artificial intelligence with a conformation description language — ConfSeq enables large language models to process 3D molecular structures by converting them into discrete token sequences.
• MultiMolecule: a modular ecosystem for biomolecular sequence-model workflows — MultiMolecule standardizes biomolecular sequence model reuse by providing complete, source-checked implementations with shared interfaces and provenance.
• From Answers to States: Verifiable Process-Level Evaluation of Chemical Reasoning in Large Language Models — Verifiable process-level evaluation, not just final answers, is crucial for assessing chemical reasoning in large language models.
• Agentic Molecular Recovery via Molecule-Aware Exploration — Molecular recovery should prioritize identity preservation and semantic alignment over mere validity restoration for invalid SMILES.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.