deepgpt at SemEval-2026 Task 1: A Chinese Humor Generation System via Instruction-Masked QLoRA and Reverse Constraint Data Mixing

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

The deepgpt team developed a Chinese humor generation system for SemEval-2026 Task 1, Subtask A, addressing the challenge of creating high-quality humor under strict text constraints, such as incorporating specific rare words or relating to news headlines. This parameter-efficient system is built upon Qwen2.5-3B-Instruct. Researchers reconstructed 8,000 multi-source Chinese jokes into a conversational instruction tuning format. A key innovation is the Instruction Masking strategy applied during 4-bit QLoRA fine-tuning, which isolates loss calculation to the target humorous text. This intervention forces the model to treat constraints as conditional inputs, effectively eradicating meaningless conversational fillers. The system achieved a hard constraint adherence (CAcc) of 94.6% and an Elo rating of 903 in official Pairwise Human Evaluation.

Key takeaway

For NLP engineers developing constrained text generation systems, consider implementing an Instruction Masking strategy during QLoRA fine-tuning. This approach, demonstrated with Qwen2.5-3B-Instruct, significantly boosts constraint adherence and eliminates conversational fillers, making it ideal for tasks requiring precise output like humor generation or factual reporting. Your models will produce more accurate and cleaner outputs under strict conditions.

Key insights

Instruction Masking with QLoRA effectively generates constrained humor by isolating loss calculation to target text.

Principles

• Isolate loss calculation to target text for strict constraint adherence.
• Treat constraints as conditional inputs, not conversational distributions.

Method

Reconstruct multi-source jokes into conversational instruction tuning format. Apply Instruction Masking during 4-bit QLoRA fine-tuning, isolating loss calculation to the humorous output.

In practice

• Use 4-bit QLoRA for efficient fine-tuning of lightweight LLMs.
• Apply Instruction Masking to mitigate formatting hallucinations.

Topics

• Chinese Humor Generation
• Instruction Masking
• QLoRA Fine-tuning
• Qwen2.5-3B-Instruct
• Constrained Text Generation
• SemEval-2026

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Automated IEP Generation from Traditional Chinese Parent-Teacher Interviews via Corpus-Grounded Feature Diffusion — Corpus-Grounded Feature Diffusion enables high-performance, privacy-preserving automated IEP generation in Traditional Chinese despite low data resources.
• QLoRA Explained: The Memory Compression Breakthrough — QLoRA enables fine-tuning large language models on consumer GPUs through efficient memory compression.
• Now in Foundry: Qwen3-Coder-Next, Qwen3-ASR-1.7B, Z-Image — New Hugging Face models offer efficient, high-performance AI for coding, multilingual ASR, and advanced image generation.
• Fine-Tuning Qwen3-VL — Fine-tuning Qwen3-VL for image-to-HTML requires careful dataset preparation and context length management.
• Simplifying the Modeling of Arbitrary Conditionals in Natural Language — AC-GPT enables arbitrary conditional modeling in causal Transformers while preserving performance and training efficiency.
• Building an LLM fine-tuning Dataset — Building multi-speaker Reddit datasets for LLM fine-tuning is complex but achievable, with efficient QLoRA methods requiring minimal samples.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.