Skip to main content
Data / ML

Designing a Production-Ready Kappa Architecture for Timely Data Stream Processing

23 January 2020 / Global
Featured image for Designing a Production-Ready Kappa Architecture for Timely Data Stream Processing
Figure 1. A simple stateful streaming job powered by a Kafka data source triggers windows every time the watermark advances. The aggregations are applied to events within the window and the result is then spilled onto the state store.
Figure 2. The same stateful streaming job as in Figure 1 in backfill mode, the only difference here being that instead of reading from an unbounded Kafka stream, we’re issuing explicit queries to Hive.
Amey Chaugule

Amey Chaugule

Amey Chaugule is a senior software engineer on the Marketplace Experimentation team at Uber.

Posted by Amey Chaugule

Category: