Skip to main content
Engineering, Backend, Data / ML, Uber AI

DataCentral: Uber’s Big Data Observability and Chargeback Platform

1 February / Global
Featured image for DataCentral: Uber’s Big Data Observability and Chargeback Platform
Image
Figure 1: Uber’s Big Data Landscape.
Image
Figure 2: User Personas of Data Platforms at Uber.
Image
Figure 3: DataCentral and Offerings.
Image
Figure 4: Clio time series historical trends.
Image
Figure 5: Application Level Yarn Queue Insights.
Image
Figure 6: HDFS Metrics Surfaced to Users and Engine Teams.
Image
Figure 7: File System Insights User Interface on DataCentral.
Image
Figure 8: Historical Insights for File System Performance.
Image
Figure 10: Contactless in Action.
Image
Figure 11: HDFS Consumption and Usage Insights.
Image
Figure 12: Example Yarn Reduction JIRA Ticket.
Image
Figure 13: High-level DataCentral architecture.
Arnav Balyan

Arnav Balyan

Arnav Balyan is a Software Engineer on the Uber Data Observability Team. He has built tools and instrumentation for providing insights into big data jobs across engines like Hive, Presto, Spark. He has worked with Massachusetts Institute of Technology in the Machine Learning domain for 2 years. His research interests and papers are focused towards anomalies in Time Series data, Brain Computer Interfaces and Opportunistic Networks.

Atul Mantri

Atul Mantri

Atul Mantri is a Senior Software Engineer on Uber's Data Platform team. He is focused on building systems that enable big data observability across all batch and real-time applications at Uber and turbocharging the cost-efficiency initiatives in the platform. Before Uber, Atul worked at Rubrik and Netapp building high-performance distributed systems. He holds a Masters degree from NC State University.

Krishna Karri

Krishna Karri

Krishna Karri is an Engineering Manager on Uber's Data Platform Team. He leads the Data Central team, which specializes in crafting advanced big data observability and chargeback platforms that are integral to the consumption reduction and cost efficiency initiatives within Uber’s Data Platforms.

Amruth Sampath

Amruth Sampath

Amruth Sampath is a Senior EM on Uber's Data Platform team. He leads the Data Analytics org comprising Hive, Spark, Flink, Pinot and DataCentral.

Posted by Arnav Balyan, Atul Mantri, Krishna Karri, Amruth Sampath