Prompts in the Wild: A Large Analyzed Collection of Transactional Prompts in Code

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Expert, medium

Summary

Prompts in the Wild: A Large Analyzed Collection of Transactional Prompts in Code" by Basmov, Goldberg, and Tsarfaty, presented at LAW XX in July 2026, introduces a dataset of 57.5K unique transactional prompts gathered from GitHub. This research posits that prompts, which directly shape Large Language Model (LLM) behavior, are linguistic objects requiring dedicated investigation. To facilitate quantitative study, the authors developed a structured ontology to transform raw, unstructured prompts into richly structured linguistic data. Analysis of this dataset revealed significant diversity in prompt usage patterns across various languages, domains, tasks, and modalities, exhibiting a typical Zipf-like distribution. The paper also details a comprehensive error analysis to validate annotation reliability and releases the dataset alongside an exploration interface.

Key takeaway

For prompt engineers or AI scientists designing LLM applications, this research highlights the importance of viewing prompts as structured linguistic data. You should consider utilizing the released dataset and ontology to analyze existing prompt patterns, informing more robust and diverse prompt engineering strategies. Understanding the Zipf-like distribution of prompt usage can guide you in exploring less common but potentially powerful prompt modalities.

Key insights

Prompts are linguistic objects that can be systematically analyzed using a structured ontology to reveal usage patterns.

Principles

• Prompts are linguistic objects.
• Prompt usage follows Zipf-like distribution.
• Structured ontologies enable empirical prompt study.

Method

Collect 57.5K transactional prompts from GitHub. Develop a structured ontology. Transform raw prompts into structured linguistic objects for quantitative analysis.

In practice

• Analyze prompt diversity across domains.
• Study prompt patterns in software.
• Explore released dataset and interface.

Topics

• Prompt Engineering
• Large Language Models
• Linguistic Annotation
• Dataset Analysis
• GitHub Data
• Ontology Development

Best for: Research Scientist, AI Engineer, NLP Engineer, AI Scientist, Prompt Engineer, Machine Learning Engineer

Related on AIssential

• Improving Alignment Between Human and Machine Codes: An Empirical Assessment of Prompt Engineering for Construct Identification in Psychology — Empirical prompt engineering, focusing on construct definition and examples, significantly improves LLM text classification alignment with human expert judgment.
• LangChain Series #3: Prompts Explained — Prompt Templates, Chat Prompts, Dynamic Prompting… — LangChain transforms basic prompts into dynamic, structured, and reusable components for robust LLM applications.
• Prompt Engineering for Named Entity Extraction from Portuguese Legal Documents — Prompt engineering with LLMs offers a viable alternative for legal NER in low-resource settings.
• Prompt Engineering: TryHackMe Challenge — Precise prompt formatting and structured examples are crucial for optimal AI performance in cybersecurity tasks.
• Context Engineering: Foundations, Categories, and Techniques of Prompt Engineering — Prompt engineering is "soft programming" LLMs by crafting inputs to guide their behavior and leverage inherent knowledge.
• Information Extraction from Electricity Invoices with General-Purpose Large Language Models — Prompt engineering is the critical lever for high-fidelity information extraction using general-purpose LLMs on semi-structured documents.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.