Migrating Your GitHub CI to Hugging Face Jobs

· Source: Hugging Face - Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Cloud Computing & IT Infrastructure · Depth: Intermediate, medium

Summary

The Trackio project successfully migrated its GitHub CI workflows to Hugging Face Jobs, demonstrating a viable alternative to GitHub-hosted runners. This migration resulted in a significant 30% reduction in CI time for CPU jobs, decreasing from 1m40s to 1m10s, and crucially enabled the execution of a new GPU test suite on actual CUDA hardware, completing in 45s on a `t4-small` instance for less than a cent. The setup leverages a custom `huggingface/jobs-actions` bridge, a dispatcher Space (e.g., `cpu-upgrade`), and a GitHub App to manage ephemeral self-hosted runners within Hugging Face's serverless infrastructure. This approach allows for precise hardware flavor selection, including `t4-small` or `h200` GPUs, and custom Docker images like `mcr.microsoft.com/playwright:v1.60.0-jammy` or `nvidia/cuda:12.4.0-runtime-ubuntu22.04`, offering enhanced flexibility and performance for machine learning projects.

Key takeaway

For MLOps Engineers or Machine Learning Engineers evaluating CI infrastructure, migrating GitHub Actions to Hugging Face Jobs offers significant advantages. You can achieve up to 30% faster CPU test execution and enable crucial GPU-accelerated testing for ML models, all while maintaining GitHub as the CI orchestrator. This approach provides granular control over hardware and Docker images, potentially reducing costs and improving debugging with CLI-accessible logs. Consider this migration to enhance your ML project's testing capabilities and efficiency.

Key insights

Migrating GitHub CI to Hugging Face Jobs cuts CPU test times by 30% and enables cost-effective GPU testing via custom runners and hardware.

Principles

Method

Duplicate a dispatcher Space, create and install a GitHub App for webhook handling, configure Space secrets, then update GitHub Actions `runs-on` labels to `hf-jobs-*` to trigger HF Jobs.

In practice

Topics

Code references

Best for: MLOps Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Hugging Face - Blog.