Skip to main content
Data / ML

Under the Hood of Uber’s Experimentation Platform

August 28, 2018 / Global
Featured image for Under the Hood of Uber’s Experimentation Platform
Figure 1. On average, over 1,000 experiments are running on Uber’s Experimentation Platform at any given time.
Figure 2. Uber’s Experimentation Platform conducts both randomized experiments and observational studies.
Figure 3. Uber’s Experimentation Platform team conducts randomized experiments by leveraging A/B/N tests to determine the lift.
Figure 4. Our XP analytics dashboard makes it easy for data scientists and other users to access and interpret their A/B test results.
Figure 5: Uber’s statistics engine is used for A/B/N experiments and dictated by fixed horizon hypothesis testing methodologies.
Figure 6. We integrate the sequential test methodology into the workflow of our XP’s outage monitoring system.
Figure 7. The sequential test methodology indicates a significant difference between our treatment and control groups, as identified in Plot B. In contrast, no significant difference is identified in Plot A.  
Figure 3. Uber’s Experimentation Platform team conducts randomized experiments by leveraging A/B/N tests to determine the lift.
Figure 9. Uber’s XP leverages continuous experiments for a variety of use cases, including hyper-parameter tuning and automated feature rollouts.
Figure 12: Our XP leverages contextual MABs for hyper-parameter tuning.
Anirban Deb

Anirban Deb

Anirban Deb is the former tech lead of the Experimentation, Segmentation, Personalization and Mobile App Development Platform data science teams at Uber and currently heading the Uber Freight data science organization.

Suman Bhattacharya

Suman Bhattacharya

Suman Bhattacharya is a senior data scientist on Uber’s Experimentation Platform team.

Jeremy Gu

Jeremy Gu

Jeremy Gu is a data scientist on Uber’s Experimentation Platform team.

Tianxia Zhou

Tianxia Zhou

Tianxia Zhou is a data scientist on Uber's Experimentation Platform team.

Eva Feng

Eva Feng

Eva Feng is a data scientist on Uber's Experimentation Platform team.

Mandie Liu

Mandie Liu

Mandie Liu is a data scientist on Uber’s Experimentation Platform team.

Posted by Anirban Deb, Suman Bhattacharya, Jeremy Gu, Tianxia Zhou, Eva Feng, Mandie Liu

Category: