Introducing Wake Vision: A High-Quality, Large-Scale Dataset for TinyML Computer Vision Applications

2024-12-02 · Source: The TensorFlow Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Internet of Things (IoT) & Connected Devices · Depth: Intermediate, short

Summary

Harvard University researchers Colby Banbury, Emil Njor, Andrea Mattia Garavagno, and Vijay Janapa Reddi have released Wake Vision, a new large-scale dataset designed to accelerate TinyML research, specifically for person detection on low-power microcontrollers and edge devices. Wake Vision, with approximately 6 million images, is nearly 100 times larger than the previous Visual Wake Words (VWW) dataset and offers two distinct training sets: Wake Vision (Large) for size and Wake Vision (Quality) for label accuracy. The dataset includes fine-grained benchmarks to evaluate model performance across real-world conditions like distance, lighting, depictions, and perceived gender/age, helping identify biases. Models trained with Wake Vision show up to a 6.6% accuracy increase over VWW and an error rate reduction from 7.8% to 2.2% with manual validation. The dataset is accessible via TensorFlow Datasets, Hugging Face Datasets, and Edge AI Labs under a CC-BY 4.0 license.

Key takeaway

For AI Scientists developing TinyML models for person detection, Wake Vision offers a critical resource to overcome data limitations. Your models can achieve significant accuracy gains and reduced error rates by leveraging this large, high-quality dataset and its fine-grained benchmarks. Consider using the combined pre-training and fine-tuning approach with Wake Vision's distinct datasets to optimize performance and identify biases early in your design process.

Key insights

High-quality, large-scale datasets like Wake Vision are crucial for advancing TinyML person detection.

Principles

• Data quality benefits under-parameterized models more than quantity.
• Combining large and high-quality datasets improves model performance.

Method

Pre-training with a large dataset and fine-tuning with a high-quality dataset yields optimal results for TinyML models, especially for person detection.

In practice

• Use Wake Vision for TinyML person detection tasks.
• Evaluate models using Wake Vision's fine-grained benchmarks.
• Submit models to the Wake Vision Leaderboard.

Topics

• TinyML
• Person Detection
• Machine Learning Datasets
• Data Quality
• Model Benchmarking

Best for: Computer Vision Engineer, AI Scientist, Research Scientist, AI Researcher, AI Engineer, Machine Learning Engineer

Related on AIssential

• [Advise] [Help] AI vs Real Image Detection: High Validation Accuracy but Poor Real-World Performance Looking for Insights — High validation accuracy does not guarantee real-world performance, indicating potential overfitting or dataset bias.
• WHAR Arena: Benchmarking the State of the Art in Efficient Wearable Human Activity Recognition — Benchmarking reveals WHAR predictive performance has plateaued, shifting focus to deployment efficiency and domain adaptation.
• Finetuning and Bulk Labelling Images with Prodigy — Fine-tuning pre-trained models with minimal annotations significantly improves image embeddings for more effective bulk labeling.
• MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models — The MMBU benchmark reveals current VLMs lack robust visual perception and domain generalization in biomedical contexts.
• From Spatial to Spectral: An Efficient, Frequency-Guided Feature Representation Learner for Small Object Detection — Shifting small object detection from spatial to spectral processing improves efficiency and accuracy by preserving high-frequency features.
• Finding Bad Image Data using UMAP and Prodigy — UMAP-driven dimensionality reduction combined with human-in-the-loop annotation effectively identifies bad examples in large image datasets.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The TensorFlow Blog.