Dropbox Collaborates with GitHub to Reduce Monorepo Size from 87GB to 20GB
Summary
Dropbox engineers, in collaboration with GitHub, successfully reduced their backend monorepo size from 87GB to 20GB, significantly improving developer productivity and continuous integration performance. The issue stemmed from inefficiencies in Git's internal compression heuristics, which produced suboptimal packfiles and disproportionately large repository growth, rather than large binaries or accidental commits. The team implemented optimized repacking strategies and adjusted Git's object delta behavior, tuning parameters with GitHub's assistance. These optimizations resulted in a 77% size reduction, clone times dropping from over an hour to under 15 minutes, and faster CI pipelines. This effort underscores the importance of treating version control systems as critical infrastructure, directly impacting engineering velocity.
Key takeaway
Dropbox, in collaboration with GitHub, reduced its backend monorepo size from 87GB to 20GB by optimizing Git's internal delta compression heuristics and server-side packing strategies. This 77% reduction cut clone times from over an hour to under 15 minutes. The approach significantly boosts developer velocity and CI pipeline performance for organizations managing large-scale monorepos.
Topics
- Monorepo Optimization
- Git Compression
- Developer Velocity
- Continuous Integration Performance
- GitHub Collaboration
Best for: CTO, VP of Engineering/Data, Director of AI/ML, Software Engineer, DevOps Engineer, MLOps Engineer
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by InfoQ.