INF-rsrs at SemEval-2026 Task 1: Is the best really better? The limits of creative work in the era of LLMs

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

INF-rsrs, a solution for SemEval 2026 Task 1: Humor Generation, tied for first place in the task ranking, demonstrating top-tier performance in creating jokes from headlines and word pairs without labeled data. The system employs a two-stage framework: a production stage and a selection stage. The production stage utilizes diverse Large Language Model (LLM) families and hyperparameter configurations to generate a wide range of candidate jokes. Each candidate is produced by an LLM prompted to act as a comedian, operating under structured constraints to ensure relevance and humor. This design aimed to substantiate the claim that directly using LLMs for creative tasks, such as humor generation, encounters an inherent limitation that cannot be overcome by simple prompting techniques.

Key takeaway

For NLP Engineers developing creative AI applications, recognize that direct LLM prompting alone may not suffice for complex tasks like humor generation. Your approach should incorporate multi-stage frameworks, leveraging diverse model families and structured prompting with specific roles and constraints to push past inherent performance ceilings. This strategy can significantly improve creative output quality and task performance, as demonstrated by INF-rsrs tying for first place in SemEval 2026 Task 1.

Key insights

Complex creative tasks like humor generation require structured, multi-stage LLM frameworks to overcome inherent limitations of simple prompting.

Principles

• LLMs hit a hard ceiling in creative tasks
• Simple prompting is insufficient for complex creativity
• Structured constraints enhance LLM creative output

Method

INF-rsrs uses a two-stage framework: a production stage with diverse LLMs and hyperparameter configurations generating candidates, followed by a selection stage. LLMs are prompted as comedians under structured constraints.

In practice

• Use multi-stage LLM frameworks for creative generation
• Employ diverse LLM models and configurations
• Prompt LLMs with specific roles and constraints

Topics

• Humor Generation
• Large Language Models
• SemEval 2026 Task 1
• Creative AI
• Prompt Engineering
• Multi-stage Frameworks

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification — Prompt sensitivity and demonstration quality critically impact LLM performance in metalinguistic classification tasks.
• IVIE: A Neuro-symbolic Approach to Incremental and Validated Generation of Interactive Fiction Worlds — IVIE's neuro-symbolic approach balances LLM creativity with symbolic consistency for coherent, engaging interactive fiction world generation.
• Re-defining Humor Data Objects for AI Humor Research — AI humor research improves by defining humor as a social interaction with context and explanations, not just a binary state.
• Chat2Workflow: A Benchmark for Generating Executable Visual Workflows with Natural Language — Automating executable visual workflow generation from natural language remains a significant challenge for current LLMs.
• How Vimeo Implemented AI-Powered Subtitles — LLMs optimized for fluency can break structural constraints, requiring architectural separation of concerns.
• Ines Montani on Natural Language Processing — Effective NLP problem-solving involves structured pipelines, precise data labeling, model fine-tuning, and leveraging LLMs for efficiency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.