Graphs of Research: Citation Evolution Graphs as Supervision for Research Idea Generation

2026-05-15 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

Graphs of Research (GoR) is a supervised fine-tuning (SFT) method that enhances large language models (LLMs) for automated research idea generation by incorporating citation-evolution graphs. Unlike existing methods that treat references as flat text, GoR extracts a 2-hop reference neighborhood for each seed paper, derives structural relations (citation position, frequency, predecessor links, publication time), and organizes them into a paper-evolution directed acyclic graph (DAG). An automated pipeline extracts data from five major ML/NLP venues, creating a dataset of 498 train, 50 validation, and 50 test seed papers with approximately 7,600 cited references. Qwen2.5-7B-Instruct-1M is fine-tuned on a structured-text prompt that includes the citation graph, edge signals, and reference information. GoR-SFT achieved state-of-the-art performance in head-to-head LLM-judge tournaments against gpt-4o-driven baselines, demonstrating the effectiveness of citation-evolution graphs as a supervision signal.

Key takeaway

For research scientists developing automated ideation systems, you should consider integrating citation-evolution graphs as a supervised fine-tuning signal. This approach, exemplified by GoR-SFT, enables smaller 7B models to outperform larger LLMs like gpt-4o on idea quality and cost-efficiency by providing structured relational context, particularly enhancing significance and clarity in generated ideas. Explore building graph-aware datasets and fine-tuning open-source LLMs to accelerate scientific innovation.

Key insights

Citation-evolution graphs, when used as an SFT signal, significantly improve LLM-based research idea generation.

Principles

• Structural relations among references are crucial for idea generation.
• Supervised fine-tuning internalizes graph signals more effectively than zero-shot prompting.

Method

Extract a 2-hop reference neighborhood, derive relations from citation context, organize into a paper-evolution DAG, serialize as structured text, then fine-tune an LLM (e.g., Qwen2.5-7B-Instruct-1M) to predict a five-field idea.

In practice

• Use 2-hop reference neighborhoods for comprehensive context.
• Annotate edges with features like role, influence, recency, and topology.
• Fine-tune smaller LLMs with graph data for cost-effective SOTA.

Topics

• Research Idea Generation
• LLM-based Idea Generation
• Citation Evolution Graphs
• Supervised Fine-tuning
• Qwen2.5-7B-Instruct-1M

Code references

• deepseek-ai/DeepSeek-V3.2-Exp
• kermitt2/grobid

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Graphs of Research: Citation Evolution Graphs as Supervision for Research Idea Generation — Citation evolution graphs provide effective supervision for LLM-based research idea generation.
• Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models — EBFT fine-tunes language models by matching sequence-level features, improving performance over next-token prediction.
• Specializing a Small Language Model for Closed-Domain Portuguese RAG using Knowledge Graph Supervision — Fine-tuning SLMs with knowledge graph supervision significantly enhances closed-domain RAG performance, surpassing larger LLMs.
• Explicit Evidence Grounding via Structured Inline Citation Generation — FullCite improves LLM attribution by jointly generating document and evidence-span citations, revealing LLMs struggle with precise span localization.
• Most Influential WWW Papers (2026-03 Version) — Citation-based ranking reveals a strong and growing influence of LLMs, RAG, and Graph Neural Networks in WWW research.
• When Do LLMs Generate Realistic Social Networks? A Multi-Dimensional Study of Culture, Language, Scale, and Method — LLM-generated social networks are significantly influenced by prompt design, culture, language, and model scale.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.