Year in Review: 2017 Highlights from Uber Open Source

Since the beginning, Uber has relied on open source technologies to build reliable, production-hardened systems that can grow at scale. In this spirit, Uber Engineering has given back several technologies to the open source community, ranging from a data visualization framework and an iOS contacts library, to a signature rendering tool and even a new programming language.  

As we ring in the New Year, let us revisit some of our most popular open source projects from 2017:  

Inspired by Dapper and OpenZipkin, Jaeger is Uber’s open source distributed tracing system written in Go. OpenTracing-compatible, Jaeger makes it easy to monitor complex, microservice-based architectures by using consistent upfront sampling with individual per service/endpoint probabilities. It is capable of monitoring distributed context propagation, distributed transaction monitoring, root cause analysis, service dependency analysis, and performance and latency optimizations.

On September 2017, Jaeger became the Cloud Native Computing Foundation’s (CNCF) 12th hosted project.

Pyro is a tool for deep probabilistic modeling, unifying the best of modern deep learning and Bayesian modeling. The goal of Pyro is to accelerate research and applications of these techniques, and to make them more accessible to the broader AI community. The language is a flexible, scalable deep probabilistic programming library (PPL) built on PyTorch. 

Pyro is a universal PPL—it can represent any computable probability distribution— and can scale to large data sets with little overhead compared to hand-written code. Agile and maintainable, Pyro is implemented with a small core of powerful, composable abstractions. Additionally, Pyro offers both automated and manual functionalities, a feat that is accomplished through high-level abstractions to express generative and inference models, while allowing experts to easily customize inference.

RIBs is the cross-platform architecture framework behind many mobile apps at Uber, including our new rider app. The name RIBs is short for Router, Interactor and Builder, which are core components of this architecture. 

In this new architecture pattern, the logic is broken into small, independently testable pieces that each have a single purpose, following the single-responsibility principle. Consequently, the RIBs  framework is designed for building mobile apps with a large number of engineers and nested states.

Named after a traditional Russian folk dance in which performers link arms in a circle, Horovod is a stand-alone Python package built on top of Uber’s Michelangelo ML-as-a-service platform. Horovod makes it easier to start—and speed up—distributed deep learning projects with TensorFlow, Uber’s preferred deep learning library. By applying a ring-allreduce approach via a Message Passing Interface (MPI) implementation, Horovod improves both the usability and performance of TensorFlow models.

For Uber’s use case, we found the MPI model to be more straightforward and require less code modifications than the distributed TensorFlow model with parameter servers. 

deck.gl 4.0 was released in April, 2017, enabling advanced geospatial exploration through a rich set of visualization layers capable of on-the-fly data aggregation. It extends deck.gl’s offerings beyond maps to non-geospatial visualizations, including machine learning and abstract data visualization (graphs, networks, etc.) use cases.

deck. gl 4.0 also makes the tool easier to use by providing more comprehensive (and interactive!) docs, standalone examples, new and advanced demos, and an upgraded guide from deck.gl 3.0.

NullAway is a fast and practical tool to help eliminate NullPointerExceptions (NPEs) in Android apps. At its core, NullAway is an open source type-based NPE checker for Java code, built as a plugin check to the Error Prone bug-finding framework.

Since implementation, NullAway has significantly improved our developers’ productivity, while maintaining the strong checking we required to deploy safely. In 2017, we contributed this tool to the open source community so others can build more reliable apps, too.

We would like to thank the open source community and all of our contributors for supporting us—we would not be able to continue growing these projects without you. To learn more about Uber’s Open Source program, visit the official Uber Open Source site, check out our GitHub page, and browse previous open source articles.

Interested in contributing? Send us a pull request. 

Brian Hsieh is Uber Engineering’s Open Source Program Manager.