Skip to main content
Engineering

Better Load Balancing: Real-Time Dynamic Subsetting

17 May 2022 / Global
Featured image for Better Load Balancing: Real-Time Dynamic Subsetting
Image
Service A makes an RPC request to Service B via on-host proxy.
Image
An example showing both A & D services calling B, with A & D co-located on the same host.
Image
The same example, but with dynamic subsetting. Notice that each proxy has a different number of outgoing connections.
Image
The above diagram shows the CPU utilization for deployments of a critical service; as it shows, all deployments converged to a similar CPU utilization.
Image
We’ve defined an imbalance indicator (the higher, the more imbalanced) based on production metrics, and the above diagram shows the change after dynamic subsetting was applied for a critical service.
Chien-Chih Liao

Chien-Chih Liao

Chien-Chih Liao is a Senior Staff Software Engineer on Uber’s Software Networking team. His contributions include traffic control, traffic load balancing, data center failover, and resilience features for Uber’s service mesh.

Pawel Krolikowski

Pawel Krolikowski

Pawel Krolikowski is a Staff Software Engineer on the Software Networking team. Before working on load balancing, he spent most of his time on the Software Networking management plane and integrations with stateful and stateless orchestration systems.

Sangeeta Kundu

Sangeeta Kundu

Sangeeta Kundu is a Senior Engineering Manager working on Uber’s Software Networking team. Over the past 3 years she has contributed to the hyper-growth phase of Uber. She spent the last 2 years leading the Software Networking Control Plane Team (traffic routing, discovery etc.) and the Multi-Zone Multi-Region Strategy at Uber for higher resilience at global scale.

Posted by Chien-Chih Liao, Pawel Krolikowski, Sangeeta Kundu

Category: