Skip to main content
Engineering

Disaster Recovery for Multi-Region Kafka at Uber

December 21, 2020 / Global
Featured image for Disaster Recovery for Multi-Region Kafka at Uber
Figure 2: Kafka replication topology in two regions
Figure 3: Architecture of active/active surge pricing
Figure 4: a. Example of cross-region message replication b. The checkpoints of the message replication
Figure 5: The architecture of active/active offset management service
Figure 6: a. Example of active/passive consumer’s failover from one region to another. The checkpoints of the message replication
Yupeng Fu

Yupeng Fu

Yupeng Fu is a Principal Software Engineer on Uber’s Data team. He leads several streaming teams building scalable, reliable, and performant streaming solutions. Yupeng is an Apache Pinot committer.

Mingmin Chen

Mingmin Chen

Mingmin Chen is a Director of Engineering and head of the Data Infrastructure Engineering team at Uber. He has been leading the team to build and operate Hadoop data lake to power multiple Exabytes of data in storage, Kafka infrastructure to power tens of trillions messages per day, and compute infrastructure to power hundreds of thousands compute jobs per day. His team builds highly scalable, highly reliable yet efficient data infrastructure with innovative ideas while leveraging many open-source technologies such as Hadoop (HDFS/YARN), Hudi, Kafka, Spark, Flink, Zookeeper etc.

Posted by Yupeng Fu, Mingmin Chen

Category: