Event Time vs Processing Time in Flink -Why it matters more than you think

2026-06-20 · Source: Data Engineering on Medium · Field: Technology & Digital — Software Development & Engineering, Data Science & Analytics, Cloud Computing & IT Infrastructure · Depth: Intermediate, quick

Summary

Apache Flink jobs often produce "confidently wrong" numbers if the distinction between event time and processing time is not understood. Every event has three timestamps: event time (when it occurred in the real world), processing time (when Flink processes it), and ingestion time (when Flink ingests it from the source). The author recounts an experience where per-minute counts in a Near Real-Time (NRT) pipeline were consistently inaccurate, off by a noticeable margin, due to relying on processing time. This discrepancy directly correlated with Kafka consumer lag. For accurate results, particularly when events arrive out of order or with delays, Flink requires event time processing, which necessitates the use of watermarks to manage event ordering and completeness.

Key takeaway

For Data Engineers building or maintaining Flink streaming pipelines, explicitly choosing event time processing is critical for data accuracy. Relying on processing time, especially with variable source lag like Kafka consumer delays, will produce consistently incorrect metrics. You must implement watermarks to correctly handle out-of-order events and ensure your analytical results reflect real-world occurrences, preventing confidently wrong downstream data.

Key insights

Relying on Flink's processing time for stream analytics can yield confidently wrong results, necessitating event time for accuracy.

Principles

• Event time ensures real-world accuracy.
• Processing time is prone to system lag.
• Watermarks manage out-of-order events.

In practice

• Implement event time for accurate counts.
• Monitor Kafka consumer lag closely.

Topics

• Apache Flink
• Event Time Processing
• Processing Time
• Watermarks
• Stream Analytics
• Kafka Consumer Lag

Best for: Data Engineer, MLOps Engineer, Software Engineer

Related on AIssential

• 7 Minutes to Understand the New Spark Streaming Feature that Changes Everything — Spark Structured Streaming's new Real-Time Mode achieves millisecond latency by optimizing data consumption, stage scheduling, and data shuffling.
• System Design Series: Apache Flink from 10,000 Feet, and Building a Flink-powered Recommendation Engine — Batch processing is a special case of streaming, enabling unified data processing engines.
• Beyond Real-Time: Why Event Processing Needs a Full Analytics Rethink — Complex Event Analytics (CEA) extends real-time detection (CEP) with AI, storage, and context to enable understanding, prediction, and action from event streams.
• Build a Real-Time Market Pulse Dashboard in Streamlit — Separate streaming, state management, and UI logic for stable, real-time market dashboards.
• Data Engineering Design Patterns — Chapter 11 — Real-time streaming data patterns are essential for modern Generative AI applications requiring instant responses.
• Article: From Batch to Micro-Batch Streaming: Lessons Learned the Hard Way in a Delta Index Pipeline — Micro-batch streaming can eliminate batch scheduling delays without requiring complex record-level processing.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Data Engineering on Medium.