Your Model Said “Cake.” I Wanted to Know Why.

2026-07-26 · Source: Deep Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Advanced, medium

Summary

Project AI Vision is a Gradio application built around a ResNet50 model trained on the Food-101 dataset, designed to address the "black box" problem in Convolutional Neural Networks. Instead of merely providing a prediction, such as "Red Velvet cake — 78.2% confidence," the tool visualizes *why* the model made its decision. It offers three distinct gradient-based visualization techniques—Standard Backpropagation, DeconvNet, and Guided Backpropagation—displayed side-by-side, along with a Pixel Inspector. This inspector allows users to click any pixel and see its exact influence on the decision, channel by channel. The system operates by performing a backward pass on a frozen model, asking how each pixel influences the predicted score, rather than updating weights. This read-only approach provides crucial insights into CNN mechanisms without altering the model's predictions.

Key takeaway

For AI Engineers or students grappling with CNN interpretability, Project AI Vision demonstrates a powerful approach to understanding model decisions. If you are building or debugging deep learning models, you should integrate gradient-based visualization techniques like Guided Backpropagation to move beyond mere confidence scores. This allows you to interrogate *which pixels* drive predictions, preventing critical failure modes and deepening your understanding of how CNNs process images. Consider implementing similar read-only explanation tools to enhance model trustworthiness and accelerate learning.

Key insights

Backpropagation can explain CNN decisions by quantifying pixel influence, not just update weights.

Principles

• CNN opaqueness is a critical failure mode.
• Explaining models doesn't require retraining.
• Gradient-based methods offer direct "why" answers.

Method

Project AI Vision uses a forward pass for prediction, then three distinct backward passes (Standard, DeconvNet, Guided Backprop) with different ReLU rules to generate gradient-based heatmaps and pixel-level influence data.

In practice

• Use Guided Backpropagation for sharpest saliency maps.
• Inspect pixel gradients for numerical "why" insights.
• Build explainability tools to deepen CNN understanding.

Topics

• Explainable AI
• Convolutional Neural Networks
• Backpropagation
• Saliency Maps
• Gradio
• PyTorch Autograd

Code references

• yokeswarans/Project-AI-Vision

Best for: Machine Learning Engineer, AI Engineer, AI Student

Related on AIssential

• 😺 🎙️ Watch: Opening AI’s black box — AI interpretability reveals models' internal "shape-like" reasoning, enabling safer, more reliable, and intentionally designed systems.
• OPTIMUS-Prime: Minimal and Sufficient Concept Explanations for Deep Vision Models — OPTIMUS-Prime provides minimal and sufficient concept-based visual explanations for deep vision models, grounded in prime implicant theory.
• Explainable Reinforcement Learning for assisting Air Traffic Controllers — Applying explainability techniques to Reinforcement Learning in safety-critical domains like Air Traffic Control is crucial for building trust.
• SketchXplain: Intuitive Visual Explanations of Image Classifiers with Sketches — SketchXplain generates intuitive, sketch-based visual explanations for image classifiers by combining saliency, concept models, and sketch optimization.
• Expresso-AI: Explainable Video-Based Deep Learning Models for Depression Diagnosis — Expresso-AI provides explainable deep learning for depression diagnosis by interpreting DCNN saliency maps from facial videos, improving predictive performance.
• On the explainability of max-plus neural networks — Linear-min-max neural networks offer inherent explainability by design, outperforming SHAP and Integrated Gradient.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Deep Learning on Medium.