Few-Shot Open-Set Audio Classification Using Attention Information-Fused Prototypes

2026-07-01 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new Few-shot Open-set Audio Classification (FOAC) method addresses the limitation of existing audio classification systems that misrecognize unseen class samples as known categories. This proposed approach can accurately classify query samples belonging to seen classes after model updates with limited support samples, while simultaneously rejecting samples from unseen classes. The FOAC model integrates a ResNet-based encoder for embedding extraction and a classifier featuring distinct prototype generators for few-shot and open-set classes. Few-shot prototypes are derived by fusing class-discriminative information from support and query embeddings, assigning higher weights to representative segments. A single prototype is generated for all open-set classes. The encoder undergoes supervised training using abundant base class samples, with base class prototypes subsequently generated under a joint loss. The classifier is then meta-trained with a small number of few-shot samples. Evaluated on LS-100, NSynth-100, and FSC-89 datasets, the FOAC method demonstrates superior AUROC and accuracy compared to prior methods, with statistically significant improvements and reduced computational complexity.

Key takeaway

For Machine Learning Engineers developing audio classification systems that must operate reliably in dynamic environments, this Few-shot Open-set Audio Classification (FOAC) method provides a robust solution. Your systems can now accurately classify known audio events while effectively rejecting novel, unseen sounds, preventing misclassification. Consider integrating its attention-fused prototype generation and meta-training approach to enhance your model's adaptability and reduce computational overhead, particularly when working with limited labeled data for new classes.

Key insights

A novel FOAC method uses attention-fused prototypes to classify known audio and reject unseen classes with few samples.

Principles

• Fuse support and query embeddings for robust prototypes.
• Weight representative embedding parts for discriminative power.
• Employ a single prototype for effective open-set rejection.

Method

Train a ResNet encoder on base classes, then generate base prototypes with joint loss. Meta-train the classifier using few-shot samples, fusing support/query embeddings for few-shot prototypes and generating one open-set prototype.

In practice

• Implement open-set audio classification for novel sound rejection.
• Explore attention-based prototype generation in few-shot tasks.
• Utilize meta-training for efficient model adaptation.

Topics

• Few-Shot Learning
• Open-Set Classification
• Audio Classification
• Attention Mechanisms
• Prototype Learning
• Meta-Training

Code references

• Jessytan/FOAC-AIFP

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Few-shot Class-variable Incremental Audio Classification via Prototype Adaptation and Pseudo Class-variable Training — The paper introduces FCIAC, a dynamic audio classification problem, solved with prototype adaptation and pseudo class-variable training for improved accuracy.
• From Spatial to Spectral: An Efficient, Frequency-Guided Feature Representation Learner for Small Object Detection — Shifting small object detection from spatial to spectral processing improves efficiency and accuracy by preserving high-frequency features.
• Attention-Based Prototype Calibration for Multi-Rater Few-Shot Medical Image Segmentation — Prototype calibration with attention effectively models multi-rater variability in few-shot medical image segmentation.
• Decoding Insect Song: A Multitask Semisupervised Orthoptera Bioacoustic Classifier — PULSE combines semi-supervised, multi-task learning with knowledge distillation to create a high-performing, domain-adapted bioacoustic classifier for Orthoptera.
• Teacher-Student Structure for Domain Adaptation in Ensemble Audio-Visual Video Deepfake Detection — A teacher-student framework enhances ensemble audio-visual deepfake detection models for robust generalization across diverse, unseen domains.
• Rethinking the Good Enough Embedding for Easy Few-Shot Learning — High-quality, off-the-shelf embeddings are sufficient for state-of-the-art few-shot classification, bypassing complex meta-learning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.