Skipper: Building Airbnb’s embedded workflow engine

2026-04-28 · Source: The Airbnb Tech Blog - Medium · Field: Technology & Digital — Software Development & Engineering, Cloud Computing & IT Infrastructure · Depth: Advanced, long

Summary

Airbnb developed Skipper, an embedded workflow engine designed to provide durable execution for critical "Tier 0" services without the operational overhead of external orchestration clusters or cloud-managed solutions. It addresses the problem of fragmented domain logic and bespoke retry systems by offering a shared library that integrates directly into existing services. Skipper leverages current infrastructure, using databases like MySQL or Airbnb's Unified Data Store for state persistence, and features a simple Java/Kotlin programming model with annotation-based contracts. It ensures workflow completion through a replay mechanism with checkpointed actions, where actions' results survive crashes and restarts, and compensation methods handle failures. This engine has been in production for over a year, powering more than 15 use cases across insurance, payments, and media processing, and has scaled to 10,000 workflows per second on Amazon DynamoDB.

Key takeaway

For Software Engineers building durable distributed systems, if minimizing external dependencies and operational overhead is paramount, consider an embedded workflow engine approach. This model allows your service to manage its own workflow processing, using existing databases and reducing single points of failure. You gain simplified development with familiar programming models, but must ensure workflow determinism and action idempotency to handle replays and potential at-least-once execution.

Key insights

Embedded workflow engines offer durable execution with minimal overhead by utilizing existing service infrastructure.

Principles

• Workflow code should mirror business logic.
• Avoid central points of failure.
• Utilize existing service infrastructure.

Method

Skipper defines Workflows for orchestration logic and Actions for individual operations. Actions are checkpointed, and durability is achieved via replay, where previously executed actions return checkpointed results instantly. Compensation methods undo effects of failed actions.

In practice

• Coordinate video processing pipelines.
• Manage Flink job lifecycles.
• Orchestrate multi-step claim processing.

Topics

• Workflow Engine
• Durable Execution
• Embedded Architecture
• Distributed Systems
• Java/Kotlin
• Idempotency

Best for: Software Engineer, DevOps Engineer

Related on AIssential

• The Agent Stack - Part 4: Runtimes, Workflows, and Durable Execution — The runtime layer is crucial for managing complex, long-running agent operations by ensuring progress and recoverability.
• Building durable Agents with Workflow DevKit & AI SDK - Peter Wielander, Vercel — Workflow DevKit simplifies building durable, observable, and resumable AI agents by abstracting complex orchestration patterns.
• Building a Production ready RAG API — Production RAG APIs require robust backend engineering to handle ingestion, streaming, rate limits, memory, and caching at scale.
• Discord Rebuilds Database Operations Around Automation to Manage ScyllaDB at Massive Scale — Discord's Scylla Control Plane automates complex distributed database operations, transforming manual, multi-day tasks into resilient, policy-driven workflows.
• Understanding Temporal — The described model for program execution involves three main components: a "worker" where the code runs, a "starter program" acting as the client to initiate the code, and a "temporal server" support…
• Write Reliable Software — Durable execution ensures software reliability by automatically managing state and recovery from failures, abstracting distributed systems complexity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Airbnb Tech Blog - Medium.