ML in a Box: Analyzing Containerization Practices in Open Source ML Projects

2026-07-14 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Cloud Computing & IT Infrastructure · Depth: Expert, extended

Summary

This study empirically analyzed 1,993 ML-related Dockerfiles from 392 open-source projects to understand containerization practices in machine learning. It found that containers serve distinct roles for training, inference, and infrastructure, with an average size of 10.27 GB and build times averaging 8.84 minutes. A significant 44.4% of commits trigger rebuilds, primarily due to context file changes (96.4%), leading to 71% wasted computation despite partial cache reuse. Training containers are notably larger (median 17.25 GB) and slower (median 14.6 min build time) than inference containers (median 1.7 GB, 1.0 min build time). Custom layers, not base images, account for 79-84% of image size. The study identified seven recurring ML-specific Dockerfile refactoring patterns that improve build efficiency and reduce container footprint.

Key takeaway

For MLOps Engineers and Machine Learning Engineers managing containerized ML workflows, you should prioritize refactoring Dockerfiles to mitigate substantial build inefficiencies. Externalize large artifacts like datasets and model weights, and reorder Dockerfile instructions to place stable dependency installations before frequently changing experiment code. Consider splitting CPU and GPU target images to reduce footprint and rebuild churn, as custom layers drive most image growth and early cache breaks waste significant build time.

Key insights

ML container builds are often inefficient due to large images, frequent rebuilds, and poor cache reuse, especially for training.

Principles

• Base images contribute minimally to ML container size.
• Cache breaks early, wasting ~70% of build work.
• Experiment commits are the largest source of wasted build work.

Method

The study used a static heuristic to estimate commit-level rebuild cost (Cache Reuse Share, CRS) by identifying the first invalidated instruction and weighting subsequent instructions based on their rebuild effort.

In practice

• Externalize datasets and model weights from Dockerfiles.
• Reorder Dockerfile steps to copy volatile files late.
• Split CPU/GPU targets into separate container images.

Topics

• Containerization
• Machine Learning Operations
• Dockerfile Optimization
• Build Performance
• Cache Invalidation
• Image Footprint

Code references

• jruokola/llm-granite-ft
• inarighas/voice-transcript
• UMassCDS/inatator
• MetrodataTeam/sherpa-grpc
• Team-Troef-Energy/SeriousGame

Best for: MLOps Engineer, Machine Learning Engineer, Research Scientist

Related on AIssential

• ML in a Box: Analyzing Containerization Practices in Open Source ML Projects — ML containerization practices are often inefficient, with significant wasted rebuild effort driven by experimentation.
• 5 Docker Best Practices for Faster Builds and Smaller Images — Optimizing Dockerfiles with specific practices yields significantly smaller images and faster builds.
• Is Your Machine Learning Pipeline as Efficient as it Could Be? — Pipeline efficiency, not just model architecture, is the primary driver of machine learning team productivity and iteration speed.
• Scaling ML Inference on Databricks: Liquid or Partitioned? Salted or Not? — Data layout, specifically file partitioning and row limits, critically impacts ML inference scalability on Databricks.
• Build an AI app with FastAPI and Docker - Coding Tutorial with Tips — Integrate Hugging Face models with FastAPI and Docker for efficient, production-ready ML applications.
• 6 Docker Tricks to Simplify Your Data Science Reproducibility — Achieve data science reproducibility by meticulously freezing Docker environments at every layer prone to drift.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.