5 categorical encodings, 5 wrong answers: one-hot, label, target, ordered target, and what CatBoost actually does

2026-05-20 · Source: Valeriy’s Substack · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, quick

Summary

The provided content introduces categorical features as a ubiquitous challenge in tabular machine learning projects, citing examples like country codes, device types, browsers, postcodes, merchant IDs, customer segments, and SKUs. It highlights their common appearance at the outset of any structured data analysis. The title further indicates an upcoming discussion on five specific categorical encoding methods: one-hot, label, target, ordered target, and the unique approach employed by CatBoost. This sets the stage for a deeper exploration of how to handle these prevalent data types effectively.

Key takeaway

For machine learning engineers starting a tabular project, be aware that categorical features like country codes or merchant IDs are a guaranteed initial hurdle. Your early data preparation efforts must account for these prevalent data types, as their proper handling is foundational for subsequent model development. Plan to dedicate significant attention to understanding and addressing these features.

Key insights

Categorical features are ubiquitous in tabular ML projects.

Topics

• Categorical Features
• Feature Encoding
• Tabular Machine Learning
• CatBoost
• One-Hot Encoding
• Label Encoding

Best for: Machine Learning Engineer, Data Scientist, AI Student

Related on AIssential

• Mastering CatBoost — CatBoost redesigns GBDTs to natively handle messy tabular data by preventing common statistical biases.
• Encoding Categorical Data for Outlier Detection — Effectively encoding categorical data into numeric formats is crucial for most unsupervised outlier detection algorithms.
• The Strategic Showdown CatBoost vs XGBoost vs — CatBoost is the mathematically strongest single-model gradient booster for large, messy tabular data.
• GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data — GOTabPFN enables TabPFN-style prediction on high-dimensional tabular data by ordering features and compressing local neighborhoods into meta-features.
• Dummy Variable Trap in Machine Learning Explained Simply — Encoding all categories with an intercept causes perfect multicollinearity, making model coefficients uncomputable.
• Top 20 CatBoost Interview Questions and Answers (Part 2 of 2) — CatBoost is a gradient boosting algorithm designed for high accuracy with minimal preprocessing, handling categorical features and overfitting robustly.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Valeriy’s Substack.