System Design: Design YouTube

2025-09-02 · Source: ByteByteGo · Field: Technology & Digital — Software Development & Engineering, Cloud Computing & IT Infrastructure · Depth: Intermediate, medium

Summary

Designing a YouTube-like video platform involves complex engineering to handle 500 hours of video uploaded every minute. The system employs pre-signed URLs for direct client-to-blob storage uploads, bypassing API servers. Large videos (1.5 GB to 30 GB) are split into 5-10 MB chunks, uploaded in parallel (e.g., six simultaneously) with SHA-256 fingerprints. A video processing pipeline converts uploaded content into 15-20 different versions, spanning resolutions from 2160p down to 240p and various codecs like H.264, VP9, and AV1, packaged in MP4 and WebM containers. This workflow is modeled as a Directed Acyclic Graph (DAG) to enable parallel processing of video segments, audio, and metadata across a worker farm, reducing processing time from hours to minutes. Video streaming utilizes adaptive bitrate streaming, where players download small, multi-second segments from CDNs, dynamically adjusting resolution based on network conditions (e.g., switching from 25 Mbps to 2 Mbps) and using HTTP range requests for instant seeking.

Key takeaway

For software engineers designing systems involving large file uploads or complex media processing, prioritize direct-to-storage uploads using pre-signed URLs to offload API servers. Implement multi-part uploads with chunking for resumability and efficiency. Model your processing pipelines as Directed Acyclic Graphs (DAGs) to enable parallel execution, significantly reducing latency for tasks like video transcoding. For media delivery, adopt adaptive bitrate streaming with CDN integration to ensure robust, high-quality playback across diverse network conditions and devices.

Key insights

System design for large-scale video platforms relies on direct uploads, parallel processing via DAGs, and adaptive streaming.

Principles

• Direct uploads keep API servers free.
• DAGs transform sequential bottlenecks.
• Adaptive streaming ensures smooth playback.

Method

Uploads use pre-signed URLs to blob storage, splitting videos into 5-10 MB chunks for parallel upload. Processing converts videos into multiple resolution/codec versions via a DAG-modeled workflow. Streaming uses adaptive bitrate with CDN-cached segments.

In practice

• Implement pre-signed URLs for large file uploads.
• Model video processing workflows as DAGs.

Topics

• System Design
• Video Streaming
• Large File Uploads
• Video Transcoding
• Content Delivery Networks
• Directed Acyclic Graph

Best for: Software Engineer

Related on AIssential

• Design a Web Crawler: FAANG Interview Question — Scaling a web crawler to billions of pages requires distributed, polite, smart, and fault-tolerant design.
• The Invisible Layer: Why the success of your data pipeline starts in the File System — Understanding file systems is crucial for optimizing data pipelines and diagnosing performance issues in data engineering.
• FAANG System Design Interview: Design A Chat System (WhatsApp, Facebook Messenger, Discord, Slack) — Designing a scalable chat system requires a hybrid communication model, layered architecture, and robust mechanisms for message delivery and presence.
• Understanding Apache Airflow: DAGs, Tasks, and Operators Explained Simply — Airflow orchestrates workflows using DAGs as blueprints, Tasks as work units, and Operators as execution templates.
• How to Build High Throughput Systems — High-throughput systems prioritize processing volume over individual operation speed, often trading latency for greater work capacity.
• Deft Scheduling of Dynamic Cloud Workflows with Varying Deadlines via Mixture-of-Experts — DEFT uses a Mixture-of-Experts DRL architecture with a graph-adaptive gating mechanism to optimize dynamic cloud workflow scheduling.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ByteByteGo.