Build an AI app with FastAPI and Docker - Coding Tutorial with Tips

2023-08-31 · Source: Patrick Loeber · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Cloud Computing & IT Infrastructure · Depth: Intermediate, extended

Summary

This content details the process of building and dockerizing a machine learning application using a vision and language Transformer model from Hugging Face, integrated with a FastAPI backend. The tutorial begins by setting up the model, which processes both images and text, requiring the Transformers library and PyTorch. A key tip shared is the use of Visual Studio Code's interactive sessions for exploring model outputs and variables, similar to Jupyter notebooks. The core of the application involves encapsulating the model's inference logic within a Python function, which is then exposed via a FastAPI endpoint. This endpoint handles image uploads and text queries, returning model predictions. Finally, the application is dockerized using the `docker init` command, which generates a Dockerfile and Docker Compose file, with minor adjustments made for Python version, Uvicorn host/port configuration, Rust compiler installation for Transformers, and user permissions to allow model downloading and saving.

Key takeaway

For Machine Learning Engineers deploying models, integrating a Hugging Face Transformer with FastAPI and Docker streamlines the path to production. Your team should leverage `docker init` for rapid containerization, ensuring to configure Uvicorn with `--host 0.0.0.0` and address specific model dependencies like the Rust compiler. This approach provides a robust, scalable, and easily deployable ML service.

Key insights

Integrate Hugging Face models with FastAPI and Docker for efficient, production-ready ML applications.

Principles

• Load ML models once, outside request functions.
• Use type annotations for data validation and autocompletion.
• Prefer regular functions over async for blocking ML tasks.

Method

Build a FastAPI app around a Hugging Face model, using VS Code interactive sessions for exploration. Dockerize with `docker init`, adjusting for dependencies like Rust and user permissions, then run with `docker compose up`.

In practice

• Use VS Code interactive sessions for model exploration.
• Employ `docker init` for quick Docker setup.
• Configure Uvicorn host to "0.0.0.0" for container access.

Topics

• Hugging Face Transformers
• FastAPI
• Dockerization
• VS Code Interactive Sessions
• Machine Learning Deployment

Best for: Machine Learning Engineer, Software Engineer, MLOps Engineer

Related on AIssential

• Docker for Python & Data Projects: A Beginner’s Guide — Docker provides reproducible environments for Python data projects, simplifying dependency management and deployment.
• What is Hugging Face? - Models, Datasets & Spaces — Hugging Face provides a unified platform for ML models, datasets, and application deployment.
• Train, Serve, and Deploy a Scikit-learn Model with FastAPI — FastAPI provides a lightweight, fast, and easy way to serve and deploy machine learning models.
• Migrating Your GitHub CI to Hugging Face Jobs — Migrating GitHub CI to Hugging Face Jobs cuts CPU test times by 30% and enables cost-effective GPU testing via custom runners and hardware.
• Building Production-Ready Agentic AI Systems with Docker and FastAPI — Agentic AI systems, capable of autonomous planning and execution, require robust production infrastructure using Docker and FastAPI.
• Day 04 of MLOps: Deploy and Serve a Machine Learning Model Using Docker and Flask — Model deployment and serving are crucial for making trained ML models accessible for real-time inference.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Patrick Loeber.