Transformer-Assisted LLM-Based Source Code Summarisation: to Enable More Secure Software Development

2026-07-23 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, quick

Summary

The "Transformer-Assisted LLM-Based Source Code Summarisation" method addresses the challenge of generating high-quality natural language summaries for source code, crucial for the Secure Software Development Lifecycle (SSDLC). While Neural Source Code Summarisation (NSCS) is vital for improving code understanding and reducing vulnerabilities, existing summaries are often missing or outdated. Current solutions include task-specific Transformer models, which excel in lexical overlap metrics but not necessarily summary quality, and code-aware Large Language Models (LLMs), which capture semantics but yield low NLG scores due to their abstractive nature. This new approach combines these methods by integrating Transformer-generated summaries into LLM prompt engineering. Experiments involving four LLMs and four distinct prompts, assisted by a task-specific Transformer, demonstrated an observed improvement of 7.8% in BLEU-4 and 5%. This technique aims to enable LLMs to produce better source code summaries for software practitioners.

Key takeaway

For Machine Learning Engineers developing code summarization tools, consider integrating Transformer-generated outputs into LLM prompts. This "Transformer-Assisted LLM-Based Source Code Summarisation" method can significantly improve summary quality, as evidenced by a 7.8% BLEU-4 increase, addressing the semantic limitations of traditional NLG metrics. Implement this hybrid approach to enhance code understanding and reduce vulnerabilities in your secure software development lifecycle.

Key insights

Integrating Transformer-generated summaries into LLM prompts enhances code summarization quality, addressing limitations of standalone methods.

Principles

• NSCS is critical for SSDLC.
• LLMs excel in semantic capture.
• NLG metrics favor lexical overlap.

Method

The method involves using task-specific Transformer-generated summaries within prompt engineering to guide Large Language Models (LLMs) in creating improved source code summaries. This was tested across four LLMs with distinct prompts.

In practice

• Enhance code understanding for maintainers.
• Reduce software bugs and vulnerabilities.
• Deploy LLMs on developer workstations.

Topics

• Source Code Summarization
• Large Language Models
• Transformer Models
• Prompt Engineering
• Secure Software Development
• Natural Language Generation

Best for: AI Engineer, NLP Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Software Engineer

Related on AIssential

• Transformer-Assisted LLM-Based Source Code Summarisation: to Enable More Secure Software Development — Combining Transformer-generated examples in LLM prompts significantly enhances source code summarization quality.
• A Large-Scale Multi-Dimensional Empirical Study of LLMs for Conversation Summarization — OmniCSEval offers a robust, multi-dimensional benchmark for evaluating LLMs in conversation summarization across diverse real-world scenarios.
• Does generative AI supersede supervised XMLC? A Benchmark Study on Automated Subject Indexing with German Scientific Literature — LLM-based generative methods offer superior graded relevance and long-tail performance for subject indexing compared to supervised XMLC.
• IEEE Rolls Out Large Language Models Virtual Training Course — Large language models are fundamental architectural components demanding deep technical understanding for reliable and secure integration into digital infrastructures.
• Balancing Usefulness and Naturalness: An LLM-based Curation Pipeline for Code Review Comments — Curating LLM training data for code review comments by balancing quality and natural diversity significantly improves model performance.
• DSLs Enable Reliable Use of LLMs — DSLs provide LLMs with constrained syntax and validation, making code generation reliable and maintainable.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.