Capturing Classic Authorial Style in Long-Form Story Generation with GRPO Fine-Tuning

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

A novel two-stage pipeline has been developed to optimize authorial style in long-form story generation, addressing challenges in subjective human evaluation and unstable automatic methods. The first stage involves training a style-similarity judge by fine-tuning a sentence-transformer with authorship-verification supervision, which then calibrates its similarity outputs into a bounded [0,1] reward. In the second stage, this judge serves as the primary reward for Group Relative Policy Optimization (GRPO), used to fine-tune an 8B story generator for style-conditioned writing. This approach bypasses the accept/reject supervision typically required by Direct Preference Optimization (DPO). The GRPO-trained 8B model achieved an average style score of 0.893 across target authors Mark Twain, Jane Austen, Charles Dickens, and Thomas Hardy, surpassing open-weight baselines. This demonstrates that AV-calibrated reward modeling offers a practical mechanism for controllable long-form style transfer, even with moderate model size and training budgets.

Key takeaway

For NLP Engineers developing style-conditioned text generation, you should consider adopting a two-stage reward modeling approach. This method, utilizing an AV-calibrated style-similarity judge with GRPO fine-tuning, allows you to achieve high style transfer scores (e.g., 0.893) on 8B models without the extensive accept/reject supervision required by DPO. This significantly reduces data annotation burdens and training costs, making sophisticated style transfer more accessible for your projects.

Key insights

AV-calibrated reward modeling enables controllable long-form style transfer for story generation without direct preference supervision.

Principles

• Style judgment can be quantified via similarity judges.
• GRPO fine-tuning avoids explicit preference data.
• Moderate models can achieve high style transfer.

Method

A two-stage pipeline: first, train a sentence-transformer as a style-similarity judge with authorship-verification, calibrating rewards [0,1]. Second, use this judge as a reward in GRPO to fine-tune an 8B story generator.

In practice

• Fine-tune 8B models for specific authorial styles.
• Develop style judges using authorship verification.
• Apply GRPO for style transfer without DPO data.

Topics

• Long-Form Story Generation
• Authorial Style Transfer
• GRPO Fine-Tuning
• Reward Modeling
• Sentence Transformers
• Authorship Verification

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Better Literary Translation: A Multi-Aspect Data Generation and LLM Training Approach — Multi-aspect iterative refinement generates superior literary translation data for LLM training.
• StyleVAR: Controllable Image Style Transfer via Visual Autoregressive Modeling — StyleVAR uses multi-scale autoregressive modeling with blended cross-attention and GRPO to achieve high-quality, controllable image style transfer.
• Drifting Preference Optimization for One-Step Generative Models — DrPO enables efficient preference finetuning for one-step generative models using black-box rewards and feature-space updates.
• Retell, Reward, Repeat: Reinforcement Learning for Narrative Theory-Informed Story Retelling — Reinforcement learning with narrative theory-informed AI feedback improves story diversity and human narrative alignment.
• Emotion-Aware Image Generation from Korean Diary Text via LLM-based Prompt Translation and LoRA Fine-Tuning — A pipeline generates emotion-aware, children's drawing style images from Korean diary text using LLM sentiment analysis and LoRA fine-tuned Stable Diffusion.
• GLASS: GRPO-Trained LoRA for Acoustic Style Steering in Zero-Shot Text-to-Speech — GLASS uses reward-trained LoRA adapters for composable, disentangled acoustic style control in zero-shot TTS.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.