Skip to main content
Engineering

No Code Workflow Orchestrator for Building Batch & Streaming Pipelines at Scale

December 11, 2020 / Global
Featured image for No Code Workflow Orchestrator for Building Batch & Streaming Pipelines at Scale
Figure 1: uWorc has a drag and drop workflow editor.
Figure 2: To update a Hive table after a workflow completes, users add an external pipeline sensor, followed by one of the many Hive Query Tasks.
Figure 3: A data science workflow that runs a notebook for offline prediction. Our tools manage and merge these predictions into a new table for wider sharing.
Image Figure 4: A Kafka task displays a list of Kafka topics for users to pick and choose from. A Hive task displays YARN queues.
Figure 5: A high-level view of the components of uWorc Architecture.
Figure 6: JSON templates help users to add new components, or to add an entirely new platform or engine with its components.
Figure 7:  Pluggable component architecture of uWorc
Figure 7: Users can build a workflow monitoring dashboard in uWorc.
Sandeep Karmakar

Sandeep Karmakar

Sandeep Karmakar is the former lead product manager with Uber’s data and machine learning platform.

Sriharsha Chintalapani

Sriharsha Chintalapani

Sriharsha Chintalapani is a Senior Staff Software Engineer and the Tech Lead for Data Platforms at Uber. The Data Quality Platform provides quality checks and alerts to our Data Assets (tables, metrics) at Uber.

Posted by Sandeep Karmakar, Sriharsha Chintalapani

Category: