The SEFR classifier

2021-01-11 · Source: Blog on Machine, Think! · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Internet of Things (IoT) & Connected Devices · Depth: Intermediate, long

Summary

The SEFR (Scalable, Efficient, and Fast classifieR) algorithm, initially published by Iranian researchers, offers a lightweight solution for machine learning on resource-constrained devices like the Arduino Uno, which features an 8-bit CPU, 16 MHz clock speed, and 2KB RAM. Implemented in Swift for iOS and Mac, SEFR is a supervised learning binary classifier that can be extended to multiclass tasks using a one-vs-rest strategy. During training, SEFR computes a weight for each feature, indicating its correlation with positive or negative classes, and a bias value to define the decision boundary. The algorithm's simplicity results in O(M*N) training time complexity and O(M) testing time, where M is features and N is examples, making it fast and memory-efficient. A demo app using Vision framework's FeaturePrint.Scene model achieved 100% accuracy on a small image classification dataset.

Key takeaway

For Machine Learning Engineers developing solutions for microcontrollers or embedded systems, SEFR offers a highly efficient and low-memory classification algorithm. You should consider SEFR when computational resources are severely limited, prioritizing speed and small footprint over peak accuracy. Its feature weighting also provides valuable interpretability for feature selection.

Key insights

SEFR is a simple, efficient, and fast classifier suitable for resource-constrained devices and interpretable feature weighting.

Principles

• Feature weights indicate importance.
• Binary classifiers extend to multiclass via one-vs-rest.

Method

SEFR trains by calculating feature weights based on average feature values for positive and negative examples, and a bias from weighted average scores, then predicts by summing weighted features and bias.

In practice

• Normalize input data to [0, 1] range.
• Use Vision framework for image feature extraction.
• Remove features with weights near zero.

Topics

• SEFR Classifier
• TinyML
• Binary Classification
• Multiclass Classification
• Feature Weights

Code references

• hollance/TinyML-HelloWorld-ArduinoUno
• hollance/sefr-swift

Best for: Machine Learning Engineer, Software Engineer, AI Student

Related on AIssential

• Expert-Guided Class-Conditional Goodness-of-Fit Scores for Interpretable Classification with Informative Missingness: An Application to Seismic Monitoring — Expert-guided, interpretable classification can outperform standard ML, especially with informative missingness and small datasets.
• Naïve Bayes: The Algorithm That’s Wrong About Everything But Right About Results — Naïve Bayes effectively classifies by combining prior class probabilities with feature likelihoods, despite a "naïve" independence assumption.
• Feature Selection Techniques in Machine Learning — Feature selection optimizes ML models by identifying and retaining only the most impactful input variables.
• AdaBoost - Explained — AdaBoost combines weak classifiers by adaptively weighting data points and their votes to form a strong ensemble.
• HiRo: A Compact Four-Directional Hierarchical Reservoir Token-Mixer for Efficient Image Classification — HiRo combines hierarchical reservoir computing and shifted windows for efficient, high-accuracy image classification with few parameters.
• Does the Data Processing Inequality Reflect Practice? On the Utility of Low-Level Tasks — Low-level data processing can improve classification accuracy for practical classifiers, despite the data processing inequality.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Blog on Machine, Think!.