Skip to main content
Engineering

Engineering a Job-based Forecasting Workflow for Observability Anomaly Detection

16 May 2018 / Global
Featured image for Engineering a Job-based Forecasting Workflow for Observability Anomaly Detection
Figure 1: uMonitor enables engineers at Uber to be notified of abnormal behavior in production.
Figure 2: Having forecasting jobs that are responsible for the same time series at adjacent time ranges retrieve their own own requisite data results in serious redundancy, burdening the metrics system.
Figure 3: By maintaining a persistent record of which forecasting jobs are either completed or in progress, F3 can infer what jobs are required in order to “fill in the gaps.” Overlaps between jobs for any given point in time, as shown here, are tolerable.
Figure 4: Allowing forecasting jobs to subselect their data post-query from a common pool minimizes unnecessary load on the metrics system due to forecasting.
Figure 6: By introducing an ability to sub-select data from a common pool of historical data, we reduce F3’s burden on the underlying metrics store by as much as 90% (figure not drawn to scale).
Jonathan Jin

Jonathan Jin

Jonathan Jin is a software engineer on Uber’s Observability Applications team in New York. He likes dry, minerally white wines, folk music, and light-roasted pour-over coffee.

Posted by Jonathan Jin

Category: