Skip to main content
Data / ML, Engineering

Presto® on Apache Kafka® At Uber Scale

April 14, 2022 / Global
Featured image for Presto® on Apache Kafka® At Uber Scale
Figure 1: Big Data Stack At Uber
Figure 2: Kafka At Uber
Image
Figure 3: Hypothetical Use Case: Check if order with UUID X missing in the Kafka Topic
Figure 4: High-level Architecture
Figure 5: Kafka Cluster/Topic and Data Schema Discovery
Image
Figure 6: Hypothetical Use Case: Check if order with UUID X missing in the Kafka Topic
Yang Yang

Yang Yang

Yang Yang is a Staff Software Engineer on Uber’s Streaming Data Team. She works on building a highly scalable, reliable Kafka ecosystem at Uber, including uReplicator, Kafka Consumer Proxy, and other internal tooling.

Yupeng Fu

Yupeng Fu

Yupeng Fu is a Principal Software Engineer on Uber’s Data team. He leads several streaming teams building scalable, reliable, and performant streaming solutions. Yupeng is an Apache Pinot committer.

Hitarth Trivedi

Hitarth Trivedi

Hitarth Trivedi is a Senior Software Engineer on Uber’s Data Analytics team. Hitarth primarily works on Presto.

Posted by Yang Yang, Yupeng Fu, Hitarth Trivedi