Building new NLP solutions with spaCy and Prodigy

2018-07-07 · Source: Explosion · Developer tools and consulting for AI, Machine Learning and NLP - Explosion.ai · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, extended

Summary

Explosion AI's co-founder Matthew Honnibal highlights that Natural Language Processing (NLP) projects frequently fail, akin to startups, due to common pitfalls in design and execution. He introduces spaCy, an open-source NLP library, and Prodigy, a commercial annotation tool, as part of a workflow designed to mitigate these risks. Honnibal argues that maximizing success requires understanding a "hierarchy of needs," prioritizing clear business process integration and robust annotation scheme design over immediate model architecture choices. He identifies a "chicken and egg problem" where product vision depends on model accuracy, which in turn requires labeled data. The proposed solution emphasizes rapid, iterative development across all project phases, from initial problem framing and data annotation to model training and evaluation, rather than a waterfall approach.

Key takeaway

For NLP Engineers designing new solutions, recognize that early project design and data strategy are more critical than model architecture. Adopt an iterative approach, using tools like Prodigy to quickly gather initial evidence from small annotation batches (e.g., 200 records) before scaling. This rapid feedback loop, combined with in-house annotation and A/B evaluation, will help you validate assumptions, refine problem framing, and significantly reduce project failure risk.

Key insights

NLP project success hinges on iterative design, data annotation, and model integration, not just model architecture.

Principles

• NLP project failure rates are high, driven by early design choices.
• Iterative development across all project phases is critical for success.
• Composing generic models is cheaper and more robust than creating new categories.

Method

Iterate on product vision, annotation schemes, data collection, and model architecture. Start with small annotation batches (e.g., 200 records) to gather evidence quickly.

In practice

• Use Prodigy for rapid text classification and fine-tuning pre-trained models.
• Employ A/B evaluation for nimble, fine-grained model comparison, even for generative outputs.
• Conduct data annotation in-house with small, consistent teams for quality and privacy.

Topics

• spaCy
• Prodigy
• Natural Language Processing
• Machine Learning Project Management
• Data Annotation
• Iterative Development
• Model Evaluation

Best for: NLP Engineer, Machine Learning Engineer, AI Engineer

Related on AIssential

• How to build resilient NLP applications — Production-grade NLP requires embracing inherent complexity, providing granular control, and integrating domain expertise for real-world value.
• The triangulation of ethical leader signals using qualitative, experimental, and data science methods — A complete spaCy-based NLP project workflow integrates data, annotation, training, and deployment.
• From a 2013 Telugu ASP.NET App to My First Open Source npm Package — 13 Years in Between — Persistent curiosity and iterative learning can transform early experiments into significant open-source contributions.
• Getting My Hands Dirty with Python and NLP — Building a practical NLP project is an effective way to learn Python libraries and data structures.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Explosion · Developer tools and consulting for AI, Machine Learning and NLP - Explosion.ai.