Questions from readers of my book

2026-03-03 · Source: Ehud Reiter's Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Advanced, medium

Summary

Ehud Reiter, author of a book on Natural Language Generation (NLG), addresses reader questions covering various aspects of NLG, from foundational concepts to practical applications and challenges. He highlights the resurgence of language grounding, a topic he explored in the early 2000s, now gaining commercial interest. Reiter discusses the role of rule-based and hybrid NLG systems in mission-critical applications, emphasizing their correctness and ease of quality assurance compared to black-box LLMs. He stresses the paramount importance of data quality over architectural factors in Neural NLG, noting that poor data leads to many LLM failures, especially in less developed countries with limited linguistic datasets. The discussion also covers evaluation based on requirements, the difficulties of maintaining both rule-based and neural systems, and risks posed by inconsistent regulatory frameworks.

Key takeaway

For CTOs and VPs of Engineering evaluating NLG solutions, prioritize data quality and consider hybrid rule-based/LLM architectures for mission-critical applications. Your teams should implement explicit content planning where possible and align evaluation metrics directly with system requirements to ensure reliability and utility. Be mindful of the significant maintenance challenges for both rule-based and neural systems, particularly the difficulty of removing obsolete knowledge from LLMs.

Key insights

Data quality is paramount for Neural NLG success, often outweighing model architecture.

Principles

• Language grounding is crucial for connecting language to real-world context.
• Rule-based NLG excels in mission-critical applications requiring correctness.
• Evaluation should align directly with system requirements.

Method

For mission-critical NLG, use hybrid systems: rules for content selection/critical parts, LLMs for expression/supporting material. Employ RAG for explicit content planning when feasible.

In practice

• Prioritize data quality in Neural NLG projects.
• Use rule-based systems for safety-critical text generation.
• Document LLM versions and QA processes thoroughly.

Topics

• NLG Systems
• Large Language Models
• Data Quality
• Rule-based NLG
• NLG Evaluation

Code references

• DCU-NLG/HEDS-3.0

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Researcher, NLP Engineer, AI Student

Related on AIssential

• Assessing LLM Reliability on Temporally Recent Open-Domain Questions — LLMs achieve high semantic alignment through paraphrasing, not lexical reproduction, challenging traditional evaluation metrics.
• Why NL-to-SQL Systems Fail: Lessons from building a Semantic SQL validation pipeline — Semantic correctness and robust validation are paramount for reliable NL-to-SQL systems, beyond mere syntax.
• NLP: From Prototype to Production — Production NLP prioritizes specific business utility and maintainability over raw academic accuracy.
• Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity — LLMs, guided by a context-aware "Complexity Score" for prompt selection, significantly outperform fine-tuned models on inferentially complex, low-prevalence data extraction.
• Cross-Lingual Empirical Evaluation of Large Language Models for Arabic Medical Tasks — English-centric LLMs exhibit significant performance gaps and reliability issues in Arabic medical tasks.
• Future of NLG evaluation — NLG evaluation must shift from artificial benchmarks to rigorous, real-world impact, qualitative, and safety assessments.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Ehud Reiter's Blog.