Kafka on Kubernetes with Strimzi vs Confluent Operator vs KRaft on Bare Metal
Summary
The article compares three primary Kafka deployment patterns: Strimzi OSS on Kubernetes, Confluent for Kubernetes (CFK) with its commercial platform bundle, and bare-metal/EC2 KRaft. It meticulously details their operational trade-offs across five key axes: cost model, latency budget, RBAC + audit surface, cluster resize story, and on-call operations surface. Strimzi, a CNCF-graduated operator, is presented as the default for greenfield Kubernetes deployments, offering Apache 2.0 licensing. CFK bundles enterprise features like Schema Registry, ksqlDB, and tiered storage, often chosen when Confluent contracts exist or specific enterprise needs arise. Bare-metal KRaft, which removed ZooKeeper in Kafka 3.5 (default in 4.0), provides superior p99 latency (1-3 ms vs. 5-10 ms on Kubernetes) and throughput, making it suitable for ultra-low-latency workloads despite higher engineering overhead. The guide emphasizes selecting a pattern based on workload-specific axis weighting rather than trends.
Key takeaway
For Senior Streaming Engineers evaluating Kafka deployments, meticulously apply the five-axis decision matrix (cost, latency, RBAC, resize, ops surface) to select the optimal pattern. Your choice between Strimzi, Confluent for Kubernetes, or bare-metal KRaft must align with specific workload requirements, avoiding generic defaults to prevent long-term operational debt and cost overruns.
Key insights
Selecting Kafka deployment (Strimzi, CFK, bare-metal KRaft) hinges on balancing cost, latency, RBAC, scalability, and operational overhead.
Principles
- KRaft mode eliminates ZooKeeper, becoming default in Kafka 4.0.
- Kubernetes adds ~200-500 μs p99 latency versus bare metal.
- Deployment choice binds for years, impacting cost and operational complexity.
Method
Evaluate Kafka deployment using a 5-question decision tree considering K8s availability, latency needs, existing Confluent contracts, OSS-first culture, and ZooKeeper migration status.
In practice
- Use "KafkaNodePool" CRDs for controller/broker role separation in Strimzi.
- Enable CFK tiered storage for topics with >7 days retention.
- Apply OS-level tuning (XFS, "vm.swappiness=1") for bare-metal p99 latency.
Topics
- Kafka Deployment
- Kubernetes Operators
- Strimzi
- Confluent Platform
- KRaft
- Low-Latency Streaming
- Total Cost of Ownership
Best for: Data Engineer, MLOps Engineer, AI Architect
Related on AIssential
See Counsel's argued verdicts on the open AI decisions leaders are weighing →
Editorial summary, takeaway, and curation by AIssential. Original article published by Data Engineering on Medium.