Growing the Data Visualization Community with deck.gl v5

Today, the Uber Visualization team open sourced deck.gl 5.3, the final deck.gl v5 release of our data visualization software. The v5 releases represent a major effort devoted to making deck.gl easier to use than ever before, and we hope that by sharing the story behind these improvements, we can get users excited to try out deck.gl v5’s new simplified APIs and features.

When we launched deck.gl v1 in 2016, we were thinking about the growing community of people who wanted to do big data visualization on the web. In particular, by open sourcing deck.gl, we invited others to use and build on the framework. We nurtured the hope of creating a collaborative WebGL visualization community that would extend beyond Uber’s own engineering organization, letting us engage with others who share our passion for visualization.

deck.gl v4 followed a phase of rapid feature development, during which we introduced support for advanced geospatial exploration along with new non-spatial visualization capabilities. But while we worked on adding features, we listened to the growing community to understand what makes adopting deck.gl appealing and what gets in the way of using it. We soon realized that in order to continue growing the deck.gl community, we needed to remove as many barriers to adoption as possible.

In v4, we introduced a slew of new demos and examples aimed at making it easier for others to start using the framework. Building on this foundation, the main goal of deck.gl v5 has been to make using the framework easier, and enabling quicker and more seamless development of WebGL-powered visualizations.

The main focus areas through which deck.gl v5 addresses this challenge are:

To make deck.gl accessible to a larger group of users, our first order of business was making it possible to use deck.gl without React. After all, the need to visualize big data on the web is clearly not unique to React developers, but previous versions of deck.gl were only compatible with React, creating a significant barrier to entry for users unfamiliar with or otherwise unable to use the React framework.

While a lot of code required modification, the most important part of making deck.gl independent from React was deciding how the new non-React-based API should look. We wanted the React, Javascript, and scripting APIs to be as similar as possible, so we settled on a “One API” philosophy, meaning that, with some few differences in how the API is initialized, almost all classes and properties have the same name and semantics across versions.

 Our “One API” approach worked for deck.gl because, even in the React version, almost the entire deck.gl API was already exposed in the form of plain, React-independent Javascript classes. Only the main DeckGL component, which was required to use the rest of the API, was a React component. We added a top level Deck Javascript class, taking exactly the same properties as the existing React component. (To keep the APIs in sync, we rebuilt the React component to be a wrapper around the Deck component.)

As any deck.gl developer can see in the code below, a non-React-based deck.gl application uses exactly the same properties as the DeckGL React version. They are just provided via the slightly different API in the new Deck class:

Through our work on the pure Javascript API in deck.gl v5, we engineered out all of the hard React dependencies from deck.gl. As a result, since deck.gl now both officially supports being used without any specific Javascript UI framework (such as React) and can now be used as a base for building integrations with other UI frameworks (such as perhaps Angular), we are able to say that deck.gl is framework agnostic. Below, we outline what framework agnosticism actually means for both new and existing users:

While framework agnostic, deck.gl is still 100 percent committed to its React integration. Although the DeckGL React component is now a wrapper on top of the new Deck Javascript class, every Javascript class in deck.gl is carefully designed to work performantly in React applications. (While a full explanation of what this means is outside the scope of this text, to get a taste of the issues involved, consider that React applications conceptually redraw the entire UI every time some part of the application state changes.)

We’d love to see deck.gl being used with other frameworks. With the availability of an official pure Javascript API, we made it much easier for application developers to do the integrations they need using their frameworks of choice and for the community to build pre-packaged versions of such integrations. While we are not currently working on these integrations, we would be happy to support such porting efforts.

The new Javascript API was a great start towards widening our user base. We now asked ourselves whether we could make deck.gl even easier to use for our developers.Even though deck.gl no longer required the use of React, we were still asking developers to set up a full web application before starting. And we know that setting up the necessary requirements for the build configuration files, package installations, etc. can be both daunting and exhausting, especially for a new user who just wants to make a quick visualization.

To make development easier, we packaged deck.gl’s new, easier-to-use React-independent Javascript API with some additional conveniences, such as automatic Mapbox base map integration, and published the resulting bundle as a script version of deck.gl.

This script version of deck.gl works out-of-the-box in CodePen and Observable environments, opening deck.gl to to more use cases such as collaborative prototyping and debugging, where a full project setup is unnecessary. Check out our ”Start Scripting with deck.gl” blog post for more detail and some exciting examples of what deck.gl scripting lets you do with a just few lines of code.

The third priority in terms of making deck.gl more accessible (after framework agnosticism and scripting support) was to further refine the API to reduce the learning curve for new users, particularly those who code to support their primary tasks rather than working as dedicated software engineers.

A specific ease-of-use aspect we targeted in the deck.gl v5 release cycle was removing the need for deck.gl applications to implement callbacks. This change was particularly compelling given that the deck.gl API already has a strong declarative flavor to it, due in large part to its React roots.

Note that the term “declarative API” refers to the ability of users to specify a framework’s behavior through a set of static properties rather than through defining dynamic behavior through functions and that declarative APIs are generally considered easier to learn and use.

So while deck.gl demos and simple applications could already be written in a near-declarative way, the need to implement callbacks for simple actions, such as tracking the window size, broke this clean flow and added clutter to the examples, giving users even more things to worry about.

To address this issue, deck.gl v5’s five new automatic features ensure that a range of small but common programming complications can now be handled trivially by deck.gl programmers:

We start off the ease-of-use list with a small but really helpful feature: GPU-based object highlighting. deck.gl is designed to be extremely performant when rendering large static data sets. However, highlighting a single object (e.g., when hovering over it with a mouse) could require updating those data sets, which could result in both a significant performance impact as well as requiring a frustrating amount of callback-style code to implement.

Enter the automatic highlighting feature, which lets programmers visually highlight an object in any deck.gl layer simply by setting the autoHighlight property to true.

We consider this feature to be an early appetizer from our ongoing General Purpose GPU (GPGPU) development track. Expect to see more GPU-powered features in future deck.gl releases. For more details about how to use this small but powerful feature and how it was implemented, check out our blog post on the topic.

Such a simple thing as keeping track of the browser’s window size added a surprising amount of extra code to our basic examples. And yet, nearly every application has to do it. To simplify this process, we added support for specifying relative sizes directly on the deck.gl component. This addition let us remove the window resize event listener clutter from deck.gl examples, as depicted below:

Note: To someone familiar with HTML and CSS size specifiers, the auto resize feature may seem minor, but it required adding a new View class system to support declarative specification of relative sizes throughout the API. Of course, the new View classes will serve many other important functions as well, but it is still a typical illustration of how simple features often need to be developed in lock-step with other things.

Before deck.gl v5, even the smallest deck.gl examples needed to remember and continuously update a view state in response to user events (which they did by defining a onViewStateChange callback). However, now most applications (that don’t need fine-grained interaction control) can just specify the initialViewState property and rely on deck.gl to automatically handle user interactions from there on:

Previously, deck.gl layers could only accept data (through their data properties) that was already loaded into the browser (often loaded from a URL serving JSON formatted data). In deck.gl v5.3, a deck.gl layer’s data property can now alternatively be set to a URL string, and deck.gl will load the data from the indicated Internet address and display it when ready. This feature allows a compatible data source to be specified with a single property, and avoids the typical 5-10 lines of callback code typically required to asynchronously load the data before instantiating the layer.

Component auto positioning allows applications to have additional HTML components, such as base maps and labels, to be automatically positioned under deck.gl viewports, which is especially powerful when using multiple views.

Finally, another feature from our ongoing GPGPU development track is deck.gl’s ability to automatically smooth the transition between different big data sets. The feature is similar to CSS transitions in classic web development, with the difference that deck.gl interpolates millions of values many times per second directly on the GPU.

As another effort to make deck.gl easier to implement for new users, we are going through the source code of our examples, carefully rewriting the source code to take advantage of the new v5 features to make sure we have not missed any opportunities (including the use of our new callback-less APIs) to make them simpler for a new programmer to digest.

While our focus has been on making deck.gl easier to implement for a broader spectrum of users (both technical and non-technical), we have of course not stopped adding visualization features.

Better late than never, the new TextLayer fills a major gap in deck.gl’s layer catalog, finally providing deck.gl users the ability to work more efficiently with text in WebGL.

The new TextLayer comes with a set of customization capabilities, such as the ability to specify fonts and character sets. And if performance of the TextLayer happens to be a concern, this showcase on the deck.gl website will hopefully lay any lingering doubts to rest.

A new Deck.views property lets the application specify multiple View classes. The new Views accept relative size parameters, enabling them to work seamlessly with the new automatic resizing feature.

With the increased use in more interactive, hybrid visualization applications, deck.gl needs to match the capabilities offered by traditional scene graph-based WebGL libraries. One of the most  requested features has been the ability to identify all objects under a particular pixel, not just the top-most object rendered on the screen.

To this end, deck.gl v5 provides a new picking function with a depth parameter that enables the application to identify multiple matches under the cursor when picking (effectively picking occluded objects).

Deck.pickMultipleObjects({x, y, depth: 10});

A new deck.gl submodule (@deck.gl/test-utils) offers automated testing support, providing strong framework support for what is often considered a blank spot in the WebGL testing community.

To more quickly respond to new requirements, deck.gl is adopting a faster monthly release schedule with a target cadence of approximately three minor monthly releases (incorporating incremental new features) followed by a major release.

We also want the deck.gl roadmap to be as open as possible, giving users a sense of what features are coming and for contributors to be able to engage with us on an equal footing.

To this end, we have published an extensive catalog of technical RFCs, enabling any user to join the design discussions around new features, and we have added a roadmap page. We are considering additional ways to enable increased participation. Please open a Github issue if you have suggestions.

The broad focus on making the deck.gl v5 framework easier to use has transformed it into a significantly more user-friendly and accessible data visualization platform.

And with the new, more flexible release cycle in combination with the opening of the deck.gl roadmap, we hope to bring more components, contributions, collaborations, and partnerships based on deck.gl to the data visualization community. We already have several big things in the works and we are looking forward to what the future holds for the deck.gl project and its growing community of contributors!

Uber’s Visualization team has open sourced the following projects: deck.gl, kepler.gl, luma.gl, react-vis, and react-map-gl. If you want to learn about all of Uber’s open source projects, check out our Github page, and, if you are interested in joining our team, visit the Uber Careers page.

Subscribe to our newsletter to keep up with the latest innovations from Uber Engineering.