Skip to main content
Data / ML, Engineering

Presto® on Apache Kafka® At Uber Scale

April 14, 2022 / Global
Featured image for Presto® on Apache Kafka® At Uber Scale
Figure 1: Big Data Stack At Uber
Figure 2: Kafka At Uber
Image
Figure 3: Hypothetical Use Case: Check if order with UUID X missing in the Kafka Topic
Figure 4: High-level Architecture
Figure 5: Kafka Cluster/Topic and Data Schema Discovery
Image
Figure 6: Hypothetical Use Case: Check if order with UUID X missing in the Kafka Topic
Yang Yang

Yang Yang

Yang Yang is a Staff Software Engineer on Uber’s Streaming Data Team. She works on building a highly scalable, reliable Kafka ecosystem at Uber, including uReplicator, Kafka Consumer Proxy, and other internal tooling.

Yupeng Fu

Yupeng Fu

Yupeng Fu is a Principal Software Engineer on Uber’s Storage, Search, and Data (SSD) team, building scalable, reliable, and performant Search and Real-Time Data Platforms. Yupeng is a maintainer of the OpenSearch Project and a member of the OpenSearch Software Foundation Technical Steering Committee (TSC).

Hitarth Trivedi

Hitarth Trivedi

Hitarth Trivedi is a Senior Software Engineer on Uber’s Data Analytics team. Hitarth primarily works on Presto.

Posted by Yang Yang, Yupeng Fu, Hitarth Trivedi