Semantic Reranking at Inference Time for Hard Examples in Rhetorical Role Labeling

2026-05-18 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new inference-time semantic reranking framework, RISE, improves Rhetorical Role Labeling (RRL) performance on challenging examples. RRL assigns functional roles to sentences in documents, crucial for legal, medical, and scientific texts. While language models (LMs) generally perform well, they struggle with low-confidence predictions on "hard examples." RISE addresses this by identifying these uncertain predictions and then reranking model outputs using contrastively learned label representations. This framework operates without requiring retraining or modification of the base LM. Evaluated across eight domain-specific RRL datasets and seven LMs (both encoder-based and causal architectures), RISE achieved an average gain of +9.15 macro-F1 points specifically on hard examples. The research also introduced manual hardness annotations to compare model and human perspectives on difficulty, showing a moderate agreement with a Cohen's kappa of 0.40.

Key takeaway

For research scientists developing or deploying LMs for Rhetorical Role Labeling, consider integrating RISE to significantly boost performance on difficult, low-confidence predictions. This framework offers a substantial +9.15 macro-F1 gain on hard examples without requiring costly model retraining or architecture changes, making it an efficient way to enhance reliability in critical applications like legal or medical text analysis. Evaluate its impact on your specific domain's challenging cases.

Key insights

RISE improves Rhetorical Role Labeling on hard examples by semantically reranking low-confidence predictions at inference time.

Principles

• Label semantics can refine LM predictions.
• Uncertainty can be addressed post-prediction.

Method

RISE identifies low-confidence predictions, then reranks model outputs using contrastively learned label representations, without model retraining.

In practice

• Apply semantic reranking to low-confidence LM outputs.
• Use contrastive learning for label representations.

Topics

• Rhetorical Role Labeling
• Semantic Reranking
• Inference-Time Optimization
• Language Models
• Hard Examples

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Revisiting Semantic Role Labeling: Efficient Structured Inference with Dependency-Informed Analysis — A new SRL framework offers explicit predicate-argument structure and 10x faster inference with modern encoders.
• Scaling Search Relevance: Augmenting App Store Ranking with LLM-Generated Judgments — LLM-generated textual relevance labels significantly improve App Store search ranking and conversion rates.
• A Self Consistency Based Reranking for Narrative Question Answering — Semantic agreement among multiple generated answers improves Narrative Question Answering robustness.
• Extracting Problem and Method Sentence from Scientific Papers: A Context-enhanced Transformer Using Formulaic Expression Desensitization — Formulaic expression desensitization and context-enhanced transformers improve problem/method sentence extraction from scientific papers, outperforming baselines.
• Slang Context-based Inference Enhancement via Greedy Search-Guided Chain-of-Thought Prompting — Slang interpretation in LLMs benefits more from structured prompting than from increased model size or temperature.
• Severity-Aware Curriculum Learning with Multi-Model Response Selection for Medical Text Generation — Severity-aware curriculum learning with multi-model response selection significantly improves medical text generation quality and relevance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.