Skip to main content
Uber logo

Schedule rides in advance

Reserve a rideReserve a ride

Schedule rides in advance

Reserve a rideReserve a ride
Engineering, AI, Data / ML

D3: An Automated System to Detect Data Drifts

February 23, 2023 / Global
Featured image for D3: An Automated System to Detect Data Drifts
Image
Figure 1: Distribution of data incidents by category
Image
Figure 2: Type of tests to detect drift in data
Image
Figure 3: D3 overall high level architecture
Image
Figure 4: Computation Layer high level architecture
Image
Figure 5: Plug-and-play anomaly detection
Image
Figure 6: Orchestrator Architecture
Image
Figure 7: Current scale of D3
Image
Figure 8: Query Optimization
Image
Figure 9: Trigger Handler at a high level
Image
Figure 10: Alerting on UI for data consumers
Image
Figure 11: Dashboard to visualize data drift
Image
Figure 12: Alert trend over time on a given dataset
Image
Figure 13: Time taken to detect incidents before and after release of D3 on marketplace datasets
Anshal Shukla

Anshal Shukla

Anshal Shukla is a Senior Staff Engineer on the Data Intelligence team. He is focused on systems that improve data reliability and simplify data consumption and production at Uber.

Vineeth Tatipathri

Vineeth Tatipathri

Vineeth Tatipathri is an engineer on the Data Intelligence team. He is one of the initial developers to work on D3 from the ground up, evolving it to scale and generating Uber wide impact.

Nipun Vats

Nipun Vats

Nipun Vats is an engineer on the Data Intelligence team at Uber. He is a main contributor to the Dataset Drift Detector framework. He is currently working on enhancing and scaling the framework to save millions of dollars.

Dinesh Jagannathan

Dinesh Jagannathan

Dinesh Jagannathan is a Senior Software Engineer on the Data Intelligence team. He is focused on building frameworks and systems in data that improve data quality, standardize best practices, and improve developer productivity.

Kousik Nath

Kousik Nath

Kousik Nath is a Senior Engineer on the Data Intelligence team. He is one of the contributors to the D3 framework. Apart from D3, he also contributes to fares related systems.

Posted by Anshal Shukla, Vineeth Tatipathri, Nipun Vats, Dinesh Jagannathan, Kousik Nath