Creating Custom Event Data Without Dictionaries: A Bag-of-Tricks

2023-04-03 · Source: Explosion · Developer tools and consulting for AI, Machine Learning and NLP - Explosion.ai · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, quick

Summary

The generation of training cases for custom event data, particularly when avoiding dictionary-based approaches, has historically been a time-intensive and laborious task. However, recent advancements in annotation methodologies, exemplified by systems such as the web-based Prodigy, are transforming this process. These newer approaches enable the rapid creation of necessary training data, significantly reducing the time and effort previously required. This efficiency gain is crucial for developing robust models for custom event detection, especially in contexts where predefined dictionaries are impractical or unavailable. This shift facilitates more agile and scalable data preparation workflows, moving away from tedious manual efforts towards streamlined, tool-assisted data generation.

Key takeaway

For NLP Engineers or Data Scientists tasked with building custom event detection models, if you are currently struggling with slow, dictionary-dependent data generation, consider adopting modern web-based annotation systems like Prodigy. This approach will significantly accelerate your training data creation, allowing you to develop and iterate on models much faster without the tedious manual effort of the past. Prioritize tools that streamline annotation to enhance project agility and model performance.

Key insights

Modern annotation systems like Prodigy streamline custom event data generation, overcoming past tedious manual efforts.

Principles

• Efficient data generation is key.
• Avoid dictionary reliance for flexibility.
• Utilize specialized annotation tools.

Method

Utilize web-based annotation systems, such as Prodigy, to quickly generate training cases for custom event data, bypassing traditional time-consuming manual or dictionary-dependent methods.

In practice

• Explore Prodigy for event annotation.
• Reduce manual data labeling time.
• Build custom event models faster.

Topics

• Custom Event Data
• Training Data Generation
• Annotation Systems
• Prodigy
• Natural Language Processing

Best for: Machine Learning Engineer, Data Scientist, NLP Engineer

Related on AIssential

• Rapid NLP annotation — Efficient, iterative, semi-automatic annotation is crucial for ML project success, enabling rapid iteration on problem modeling, not just model parameters.
• Training a Named Entity Recognition Model with Prodigy and Transfer Learning — Efficient NER model training for specific domains is achieved through semi-automatic annotation and transfer learning with tools like Prodigy.
• EVENT5Ws: A Large Dataset for Open-Domain Event Extraction from Documents — EVENT5Ws is a large, manually verified open-domain event extraction dataset for developing generalizable algorithms.
• Promoting Simple Agents: Ensemble Methods for Event-Log Prediction — N-gram models offer competitive accuracy for event-log prediction with significantly fewer resources than neural networks.
• How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data — Structured pedagogical formats and diverse outputs are key to high-quality synthetic pretraining data, not larger generator models.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Explosion · Developer tools and consulting for AI, Machine Learning and NLP - Explosion.ai.