ThinkLog: Leveraging Reasoning for Log Statement Generation

2026-01-22 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

ThinkLog is a novel LLM-based method designed for end-to-end log statement generation, addressing the limited accuracy of existing approaches. Developed by researchers at Kyushu University, ThinkLog incorporates explicit reasoning into prompts using few-shot examples to guide LLMs in deciding log insertion locations, severity levels, and message content. This method constructs a "reasoning pool" from real-world software examples, storing rationales for logging decisions. Evaluated on 9,619 Java methods from public GitHub repositories, ThinkLog achieved 20.55% log statement generation accuracy, marking a 15.4% improvement over the best existing method, UniLog. Furthermore, these accuracy gains were realized at approximately 50% of the online inference cost compared to UniLog, demonstrating both effectiveness and cost-efficiency. ThinkLog also showed stronger generalization on unseen code, with a 20.7% performance degradation compared to UniLog's 30.2%.

Key takeaway

For Machine Learning Engineers developing automated logging tools, you should integrate explicit reasoning into your LLM-based generation pipelines. By constructing a reusable reasoning pool that captures "why" logging decisions are made, you can achieve higher accuracy and better generalization on unseen code, while also reducing online inference costs by approximately 50%. Consider using 1-shot prompting with sufficiently detailed reasoning to maximize benefits without excessive cost.

Key insights

Incorporating explicit reasoning into LLM prompts significantly improves end-to-end log statement generation accuracy and cost-efficiency.

Principles

• Explicitly model "why" for logging decisions.
• Reasoning pool enhances LLM consistency.
• Sufficient reasoning length is crucial.

Method

ThinkLog builds a reasoning pool from ground-truth log statements, then uses retrieved reasoning-augmented few-shot examples for LLM warmup and inference to generate log statements.

In practice

• Construct a reasoning pool with decision rationales.
• Use 1-shot prompting with informative reasoning.
• Explicitly cover position, level, and message in reasoning.

Topics

• Log Statement Generation
• Large Language Models
• Reasoning Chains
• Few-shot Learning
• Software Maintenance
• Java Programming

Code references

• google/google-java-format

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Software Engineer

Related on AIssential

• Teaching Reasoning Models When to Stop: Data, Rewards, and Self-Aware Decoding — Efficient CoT reasoning requires balancing length reduction with accuracy, often by leveraging model confidence.
• Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs — Code-on-Graph integrates LLMs with KGs using programmatic reasoning, representing schemas as Python classes for flexible, scalable knowledge utilization.
• Thinking with Reasoning Skills: Fewer Tokens, More Accuracy — Distilling and retrieving reusable reasoning skills significantly improves LLM efficiency and performance.
• SpecMind: Cognitively Inspired, Interactive Multi-Turn Framework for Postcondition Inference — SpecMind enables LLMs to iteratively refine program specifications through interactive, feedback-driven reasoning.
• Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning — Selective verification optimizes reasoning by deciding when to re-evaluate, reducing cost and harmful answer changes.
• Domain-Specialized Tree of Thought through Plug-and-Play Predictors — DST enhances LLM tree-based reasoning by using a lightweight, adaptive predictor for efficient, context-aware branch pruning.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.