Prompting Complexity: Shortest Prompts for Texts and Behaviors in LLMs

2026-07-07 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

This paper introduces "prompting complexity," a novel metric for instruction-tuned language models, defining it as the shortest plausible prompt required for deterministic decoding to produce a specific target text. This concept serves as an LM-relative analogue to resource-bounded Kolmogorov complexity, where the prompt acts as a program and the model interface as the interpreter. Unlike classical Kolmogorov complexity, this measure is intentionally non-universal and highly model-dependent, meaning a text's cost can vary significantly across different models. The definition restricts prompts to plausible human-readable texts, aligning with prompt engineering practices. The authors extend this to "soft prompting complexity" for approximate outputs, framing it as a lossy text compression and a target for prompt optimization. They also define "prompting distance" and "behavioral prompting complexity," establishing a formal research agenda to empirically study which texts and behaviors are accessible via short, plausible prompts.

Key takeaway

For Prompt Engineers and AI Scientists focused on optimizing LLM outputs, this framework offers a formal method to quantify prompt efficiency and model-specific text accessibility. You should consider using prompting complexity concepts to systematically evaluate how short, plausible prompts influence deterministic or approximate text generation. This can guide efforts to refine prompt engineering strategies and compare the inherent "cost" of generating specific content across different instruction-tuned models.

Key insights

Prompting complexity quantifies the shortest human-readable prompt needed for an LLM to deterministically produce a target text.

Principles

• Prompting complexity is model-relative.
• Prompts must be plausible human-readable texts.
• Model weights supply omitted prompt information.

Method

The paper defines a formal framework for measuring prompt efficiency and accessibility, extending to approximate outputs and behavioral specifications for LLMs.

In practice

• Optimize prompts for specific LLMs.
• Empirically study text accessibility via short prompts.

Topics

• Prompting Complexity
• Large Language Models
• Prompt Engineering
• Kolmogorov Complexity
• Text Generation
• Model Evaluation

Best for: Research Scientist, AI Engineer, NLP Engineer, AI Scientist, Machine Learning Engineer, Prompt Engineer

Related on AIssential

• Explain Like I'm 5 or Whatever I Choose: Evaluating the Interactive Potential of Language Model Responses — LLMs inconsistently adjust response complexity, failing to reliably differentiate jargon and information across user-controlled levels.
• Prompt Engineering: Saying what matters most — Effective prompts are critical for guiding LLMs, defining behavior, controlling output quality, and managing operational costs.
• How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data — Structured pedagogical formats and diverse outputs are key to high-quality synthetic pretraining data, not larger generator models.
• Prompt Repetition: Google's Surprisingly Strong Baseline for Non-Reasoning LLMs — Repeating prompts can dramatically improve LLM accuracy by providing a "clean second read" with full context.
• Spurious Prompts: Can Irrelevant Prompts Steer Large Language Models? — Large Language Models exhibit surprising sensitivity to semantically irrelevant "spurious prompts," which can systematically steer their behavior.
• Less Is More: Cognitive Load and the Single-Prompt Ceiling in LLM Mathematical Reasoning — LLM mathematical reasoning accuracy plateaus despite extensive prompt engineering, limited by undecidability and prompt complexity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.