Syntactic Priming in Few-Shot Learning: How Demonstration Structure Shapes LLM Performance

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

A study investigated syntactic priming in large language models' (LLMs) few-shot learning, examining how demonstration example structure influences outputs. Researchers tested four model families (Llama, Mistral, Qwen, Gemma) across four syntactic constructions: passive voice, cleft sentences, dative alternation, and particle placement. Results showed robust syntactic priming effects, with models 1.3x to 6.4x more likely to produce constructions matching demonstration syntax, depending on the type. Priming strength positively correlated with model size (r = 0.85, p = 0.068), intensifying from 7B to 14B parameter models. The study found priming to be construction-specific, not general stylistic preference, and persistent across multiple intervening sentences. Syntactic structure in demonstrations influenced output style across sentence completion, paraphrase generation, and story continuation tasks, even when a different syntactic form was expected. A new benchmark, SyntaxPrime-ICL, was released.

Key takeaway

For prompt engineers designing few-shot examples, understand that the syntactic structure of your demonstrations profoundly influences LLM output. Your chosen syntax can bias models to produce specific constructions 1.3x to 6.4x more often, potentially overriding the desired output format for a given task. You should carefully review and control the syntactic patterns in your prompts, especially for larger models, to ensure outputs align with task requirements rather than unintended priming effects.

Key insights

LLMs exhibit strong syntactic priming from few-shot demonstrations, influencing output structure beyond surface-level matching.

Principles

• LLM syntactic priming strength varies by construction.
• Priming effects intensify with LLM parameter count.
• LLMs encode syntactic abstractions beyond surface matching.

Method

Systematic experiments across four LLM families (Llama, Mistral, Qwen, Gemma) using four syntactic constructions to measure output matching demonstration syntax.

In practice

• Few-shot prompt syntax dictates LLM output style.
• LLMs may override task-specific syntax with primed forms.
• Evaluate syntactic priming with SyntaxPrime-ICL benchmark.

Topics

• Few-shot Learning
• Large Language Models
• Syntactic Priming
• Prompt Engineering
• Psycholinguistics
• SyntaxPrime-ICL

Best for: AI Engineer, Machine Learning Engineer, Research Scientist, AI Scientist, NLP Engineer, Prompt Engineer

Related on AIssential

• Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification — In-context learning for Turkish LVCs is highly prompt-sensitive, with few-shot demonstrations improving LLM calibration to match supervised baselines.
• Analysis of the Neglect-Zero Effect in Large Language Models — LLMs may not share the human "neglect-zero effect" cognitive bias regarding empty sets.
• Multilinguality of Large Language Models From a Structural Perspective — LLMs' multilinguality involves structural differences, with low-resource languages diverging more from English.
• Scaffold, Not Vocabulary? A Controlled, Two-Tier, Pre-Registered Study of a Popperian Code-Generation Skill — Prompt scaffold structure, not specific Popperian content, drives code generation correctness, while LLM judges exhibit significant bias.
• [R] Empirical evidence for a primitive layer in small language models — 18 experiments across 4 architectures — Small language models exhibit a consistent activation gap between scaffolding and content semantic primitives.
• Spurious Prompts: Can Irrelevant Prompts Steer Large Language Models? — Large Language Models exhibit surprising sensitivity to semantically irrelevant "spurious prompts," which can systematically steer their behavior.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.