Skip to main content
Backend, Engineering

Building Uber’s Fulfillment Platform for Planet-Scale using Google Cloud Spanner

September 29, 2021 / Global
Featured image for Building Uber’s Fulfillment Platform for Planet-Scale using Google Cloud Spanner
Figure 1: Application architecture components of the new Fulfillment Platform.
Figure 2: High-level LATE architecture.
Figure 3: The Uber-GCP network infrastructure.
Figure 4: Benchmarked results for stable and failed over state network operations.
Figure 5: Transaction Analyzer Workflow to detect interfering transactions.
Figure 6: Comparison of without TCP RST optimization (left) vs with TCP RST optimization (right).
Figure 7: Comparison of without graceful reconnect (left) vs with graceful reconnect (right).
Figure 8: End-to-End Auto-Scaler workflow to tune cluster node configuration.
Figure 9: Read and Write interactions with On-Prem Cache and Cloud Spanner.
Ankit Srivastava

Ankit Srivastava

Ankit Srivastava is a Principal Engineer at Uber. He has led and contributed to building software that scales to millions of users of Uber across the world. During the past 2 years, he led the ground-up redesign of Uber's Fulfillment Platform that powers the logistics of physical fulfillment for all Uber verticals. His interests include building distributed systems and extensible frameworks and formulating testing strategies for complex business workflows.

Fabin Jose

Fabin Jose

Fabin Jose is a Senior Network Engineer II on the Global Network Infrastructure team at Uber. He has led and contributed to Datacenter Network Design, Datacenter builds and Network Software Automation efforts. His current focus areas include Cloud Networking and Network reliability improvement initiatives at Uber.

Jean He

Jean He

Jean He is a Senior Staff Network Engineer at Uber. She has led, drove, and contributed to the technology roadmap, architecture, and design of Uber’s data center, backbone, and cloud network infrastructure to provide cost-effective, reliable, and optimal production network foundation to serve Uber business across the world. In the last two years, she has led the hybrid network architecture serving the rewrite of the Fulfillment architecture.

Nandakumar Gopalakrishnan

Nandakumar Gopalakrishnan

Nandakumar Gopalakrishnan is a Senior Software Engineer II at Uber. He is currently leading multiple projects in the Fulfillment Framework and Persistence; building some of the foundational components that are responsible for fulfilling an Uber trip at scale. Over the past two years, he has driven and contributed to building Uber's fulfillment platform on top of Spanner using Uber’s network topology.

Ramachandran Iyer

Ramachandran Iyer

Ramachandran Iyer is an Engineering Manager at Uber. He is a technical architect with experience in designing and delivering large-scale, mission-critical platforms and distributed systems. He has led the development of Spanner Client and Fulfillment integration, re-architecting real-time search and indexing platforms and is currently working on the storage infrastructure at Uber. His interests include distributed systems, databases, storage, and search platforms.

Uday Kiran Medisetty

Uday Kiran Medisetty

Uday Kiran Medisetty is a Principal Engineer at Uber. He has led, bootstrapped, and scaled major real-time platform initiatives in his tenure at Uber. He worked on revamping the overall Mobile API architecture from a monolithic polling-based flow to a bidirectional streaming-based flow. In the last 2 years, he led the re-write of the Fulfillment architecture that powers mission-critical business flows for all Uber verticals applications in the company. In the last couple of years, he led the re-architecture of Uber’s core fulfillment platform.

Posted by Ankit Srivastava, Fabin Jose, Jean He, Nandakumar Gopalakrishnan, bowie@uber.com, Ramachandran Iyer, Uday Kiran Medisetty