Towards a Linguistic Evaluation of Narratives: A Quantitative Stylistic Framework

2026-04-22 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

Alessandro Maisto proposes a quantitative framework for evaluating narrative quality by analyzing linguistic features. This methodology extracts 33 quantitative linguistic features, categorized into lexical, syntactic, and semantic groups, to assess narrative quality automatically. An experiment conducted on a corpus of 23 books, comprising both canonical masterpieces and self-published works, demonstrated the model's ability to cluster narratives effectively, distinguishing almost perfectly between professionally edited and self-published texts. The framework was further validated against a human-annotated dataset, where it significantly outperformed traditional story-level evaluation metrics, confirming the efficacy of quantitative linguistic features in objectively assessing narrative quality.

Key takeaway

For research scientists developing automated content evaluation systems, this work suggests that focusing on quantitative linguistic features can yield more objective and effective narrative quality assessments than traditional subjective metrics. You should consider integrating a comprehensive set of lexical, syntactic, and semantic features into your models to improve the accuracy of distinguishing between professionally edited and self-published content, potentially streamlining editorial workflows.

Key insights

Linguistic features can quantitatively assess narrative quality, distinguishing professional from self-published works.

Principles

• Narrative quality has quantifiable linguistic indicators.
• Linguistic features can differentiate editorial standards.

Method

The method involves extracting 33 quantitative linguistic features (lexical, syntactic, semantic) from narratives, then using a similarity matrix to cluster texts and evaluate against human annotations.

In practice

• Automate initial screening of narrative submissions.
• Identify linguistic patterns in high-quality texts.

Topics

• Quantitative Stylistic Framework
• Narrative Evaluation
• Linguistic Features
• Text Quality Assessment
• Computational Linguistics

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Towards a Linguistic Evaluation of Narratives: A Quantitative Stylistic Framework — Linguistic features can quantitatively assess narrative quality, distinguishing professional from self-published works.
• Characterizing Narrative Content in Web-scale LLM Pretraining Data — Narrative structure in web-scale LLM pretraining data is measurable and unevenly distributed, impacting model reasoning.
• Which sentence is doing the most work in your favourite novel ? I tried to find out. — Machine learning can identify semantically distinctive sentences in texts, revealing structural or narrative pivots.
• Auditing the Evaluators: How Far Can Automatic Evaluation Go in Assessing Portuguese Financial Texts? — Automatic and LLM-as-a-judge metrics struggle with Portuguese financial texts, requiring domain-specific evaluation.
• From Comments to Trends: Studi NLP dan Computational Linguistics dalam Mendeteksi Topik Viral dari… — NLP and computational linguistics are crucial for detecting social media trends from vast, informal, and diverse comment data.
• A Cross-Genre Analysis of Discourse Relation Signaling in the GUM Corpus — The paper analyzes discourse relation signaling across diverse genres in the GUM Corpus.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.