Skip to main content
Engineering

Introducing AthenaX, Uber Engineering’s Open Source Streaming Analytics Platform

October 9, 2017 / Global
Featured image for Introducing AthenaX, Uber Engineering’s Open Source Streaming Analytics Platform
Figure 1:  AthenaX takes in streaming data and queries as inputs, computes the results, and then pushes them to a variety of outputs.
SELECT
   HOP_START(rowtime, INTERVAL ‘1’ MINUTE, INTERVAL ’15’ MINUTE) AS window_start,
   restaurant_uuid,
   COUNT(*) AS total_order
FROM ubereats_workflow
WHERE state = ’CREATED’
GROUP BY
   restaurant_uuid,
   HOP(rowtime, INTERVAL ‘1’ MINUTE, INTERVAL ’15’ MINUTE)
CREATE FUNCTION AirportCode AS;
SELECT
   AirportCode(location.lng,location.lat) AS airport
   driver_id AS driver_id,
   
FROM
   event_user_driver_app
WHERE
 NAME =‘trip_start’
SELECT
   w.created_timestamp,
   w.datestr,
   w.restaurant_uuid,
   w.order_job_uuid,
   o.price,
   o.currency,
FROM
   ubereats_workflow_etd_summary w
JOIN
   ubereats_order_state_changes o
ON
   o.job_uuid = w.order_job_uuid
WHERE
   w.status IN (‘CANCELED_BY_EATER’, ‘UNFULFILLED’)
AND
   w.proctime
 BETWEEN
   o.proctime INTERVAL ’60’ SECOND
 AND
   o.proctime + INTERVAL ’60’ SECOND
Figure 2: The AthenaX compilation process is composed of a series of DAGs and nodes. Each DAG describes the data flow of the queries, and each node describes the task to be performed when the data flows through it. Figure 2(a), 2(b) and 2(c) show the original logical plan, the optimized logical plan, and the compiled data flow program in Flink. (The physical plan is omitted for brevity as it is almost identical with Figure 2(c)).
Uber’s Streaming Analytics team takes a break from working on AthenaX to smile for the camera. Back row: Bill Liu, Ning Li, Jessica Negara, Haohui Mai, Shuyi Chen, Haibo Wang, Xiang Fu, & Heming Shou. Front row: Peter Huang, Rong Rong, Chinmay Soman, Naveen Cherukuri, & Jing Fan.
Naveen Cherukuri

Naveen Cherukuri

Naveen Cherukuri is a senior software engineer on Uber's Big Data team.

Posted by Haohui Mai, Bill Liu, Naveen Cherukuri

Category: