Skip to main content
Engineering, Backend, Data / ML

How Uber Accomplishes Job Counting  At Scale

22 May / Global
Featured image for How Uber Accomplishes Job Counting  At Scale
Image
Figure 1: User Tenure Store Architecture Diagram.
Image
Figure 2: Apache Pinot™ Query Failures due to Broker Timeout.
StatisticBefore Bloom FiltersAfter Bloom Filters
timeUsedMs3871740
numDocsScanned2121
totalDocs50,520,067,05350,550,326,486
numServersQueried1818
numServersResponded1818
numSegmentsQueried20,49120,488
numSegmentsProcessed4,82948
numSegmentsMatched1717
numConsumingSegmentsQueried22
numEntriesScanned16,466,904147
Figure 3: Query Statistics before, and after implementing bloom filters via Pinot Query Console.
Image
Figure 5: With and without Bloom Filter Performance Comparison.
Redispatched rides (Another driver is dispatched to fulfill an already assigned job)To whom should the job tenure be attributed to? Both drivers? If these are represented as multiple records, should it count as 2 jobs for the rider?
Guest RidesDue to the nature of these accounts, they can cause hot shards, causing table scan queries to be expensive.
Unfulfilled ordersShould unfulfilled orders contribute to tenure?
Requester cancellation, Order failure, etc.
Multiple dispatch of same provider on the same order1 order with multiple jobs fulfilled by the same driver should contribute to how much tenure for the requester?
Scheduled ordersShould orders that haven’t occurred yet accrue tenure against a subject?
Ryan Woo

Ryan Woo

Ryan is a Senior Software Engineer on the Safety and Insurance Team, based in San Francisco.

Sameer Kapoor

Sameer Kapoor

Sameer is an Engineering Manager on the Safety and Insurance team, based in San Francisco.

Posted by Ryan Woo, Sameer Kapoor