CoreELM: An Open-Source Framework for Aligning Large Language Models to Embedding Spaces

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Expert, quick

Summary

CoreELM is an open-source, domain-agnostic framework designed to align Large Language Models (LLMs) with embedding spaces, utilizing the Embedding Language Model (ELM) method. Developed by Brian Ondov et al. and presented at BioNLP 2026, this framework addresses limitations in interpreting, exploring, and reversing embedding spaces. The authors demonstrate CoreELM's capabilities by training models to recover, summarize, and compare clinical trial abstracts solely from their embeddings. The framework shows improved reliability in inverting embeddings to text compared to existing methods. Furthermore, CoreELM can decode novel, interpolated embeddings into new clinical trial abstracts that human experts cannot distinguish from authentic ones. These generated abstracts also respond to concept vectors, such as age and sex of study subjects, enabling controlled text generation.

Key takeaway

For NLP Engineers and Research Scientists exploring LLM alignment or controlled text generation, CoreELM offers a robust open-source solution. You can use this framework to reliably invert text embeddings back into human-readable text, enhancing transparency. Furthermore, you can generate novel, high-quality text by interpolating embeddings and manipulate generated content by moving embeddings along specific concept vectors, like age or sex, for targeted applications.

Key insights

CoreELM provides an open-source framework for aligning LLMs to embedding spaces, enabling reliable text inversion and controlled generation.

Principles

• Aligning LLMs to embeddings enhances interpretability.
• Embedding spaces can be inverted to generate text.
• Concept vectors enable controlled text generation.

Method

The CoreELM framework aligns LLMs to embedding spaces using the ELM method, training models to recover, summarize, and compare text from embeddings, and to decode interpolated embeddings into novel text.

In practice

• Invert text embeddings back to original text.
• Generate novel text from interpolated embeddings.
• Control generated text via embedding concept vectors.

Topics

• Large Language Models
• Text Embeddings
• Embedding Language Model
• Open-Source Framework
• Biomedical NLP
• Controlled Text Generation

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation — Specialized text embedding models can be effectively adapted for low-resource languages using existing multilingual foundations.
• scLLM-DSC: LLM-Knowledge Enhanced Cross-Modal Deep Structural Clustering for Single-Cell RNA Sequencing — Integrating LLM-derived biological semantics with structural transcriptomic features enhances scRNA-seq clustering accuracy and interpretability.
• ECA: Efficient Continual Alignment for Open-Ended Image-to-Text Generation — ECA enables efficient continual alignment in OpenITG by adapting VLM alignment modules to evolving visual data without prior raw data.
• SciText2Eq: Assessing LLMs for Explainable Equation Generation for Scientific Creativity — LLMs moderately generate equations from text but struggle with semantic accuracy and reliable self-evaluation.
• DALM: A Domain-Algebraic Language Model via Three-Phase Structured Generation — DALM uses domain algebra and structured denoising to prevent cross-domain contamination and hallucination in language models.
• Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings — LLM unembedding matrices encode an "edge spectrum" subspace biasing embeddings towards frequent, uninformative tokens, which EmbedFilter removes.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.