At a company as large and as complex as Uber, the volume of internal communication can easily become overwhelming. Our employees use multiple independent systems to send and receive a variety of notifications every single day. These include:
- Pullo – to raise access requests for software tools and services
- Uber Feedback – to provide and receive co-worker feedback
- Uber Learning – to undertake assigned training
- Workday – to apply for leaves, sign HR contracts, etc.
- ERD-PRD tool – to seek and provide approval of engineering and product documentation
- UberHub – to make requests for IT infra, hardware & software, etc.
Thus employees receive several action items and approval requests from multiple applications on a daily basis. Unified Action Platform or uAct has been built with a view to help employees keep on top of their assigned tasks and action items. uAct aggregates all such requests into one place for employees to easily view and address.
Employees go through several applications every day to submit, track, and approve requests. Notifications are sent to request approvers via email. These can easily be lost in a barrage of emails, increasing the turnaround time for requests and a lot of follow-ups! This results in delays, breaches of deadlines, or compliance issues for the company.
At Uber, we typically have regular notifications generated from at least 20 systems, thus making it difficult to manage the enormous volume of requests. Currently, there is no way to view all pending requests in one place.
uAct was created with the objective to:
- Provide an aggregated view of an employee’s pending requests across several applications
- Track, manage, and take action on requests
- Approve/reject requests on the move via Slack
- Nudge the approver in case of critical or long-pending actions
- Filter/sort requests on various parameters to quickly get to the important ones
- Maintain an audit trail of actions initiated from uAct
uAct aggregates pending actions/ approvals/ requests from multiple applications in two ways:
- Real-time Ingestion: Requests are fetched in real-time from source applications
- Offline Ingestion: Requests are fetched from source applications via cron jobs and stored in uAct’s data store using the Apache Camel ™ framework. Request Crawler fetches either in an incremental way (to get updated requests) or a complete way (to get all requests). These crawls are set up at scheduled intervals that could be different for each application based on the nature of the data being ingested. Request Ingestor saves the fetched requests into the Requests Store.
View Requests in uAct:
In order to render requests in the UI, the Request Provider pulls requests from real-time source applications and merges it with the relevant requests from the offline-ingested Requests Store.
There are 2 main views in uAct’s user interface:
- Received Requests View: This is uAct’s default view showing requests that are assigned to the user to take some action, such as to Approve/Reject
- Sent Requests View: This view shows requests initiated by the user and helps in tracking them to closure
uAct’s UI allows users to save their default view preferences. These user preferences are saved in the User Preferences Store.
Based on the APIs provided by the source applications, users can act on the requests they see in two ways:
- Be redirected to the source application, landing on the deep-linked request. The user can then take the desired action in the source application itself.
- Take action within the uAct Platform. Action Dispatcher authenticates and authorizes the request, then relays the action (e.g., Approve/Reject) to the source application. Additionally, Audit Trail Provider creates an entry in the Audit Store for compliance and visibility purposes.
uAct incrementally onboards new source applications each with multiple configurations. These are maintained in the Config Store (deployed independently and changed dynamically). It contains configuration details like whether the application is enabled for that user; what actions are allowed for this application, whether data is ingested offline or needs to be fetched in real-time, timeout values, etc.
Experimentation or rollout of features is controlled via Flipr. Flipr creates a feature flag that helps in the rollout of features to a specific set of users, toggling features on/off, etc.
uAct has analytics, insights, alerting, and monitoring mechanisms for debugging issues. Multiple metrics from UI and backend for errors and insights are captured, which is managed via a central observability platform. Failure metrics have been set up to instantly detect and alert on issues. Grafana® dashboards are also created on top of these metrics for analytics and insights. uAct UI has been integrated with Google Analytics to track the number of active users, click events, and other behavioral insights from end users.
Backend Low-Level Design
The backend of uAct uses the Onion architecture. It provides loose coupling and high cohesion, thus helping in easy integration with new source applications.
A brief description of its components is as follows:
- Receives the requests coming to the platform
- Converts the requests object to the entity object and transfers it to the next layer (controller)
- Returns the results received from the next layer (controller) into the response object
- Contains the business logic of the application
- Fetches the data from the gateway and repository orchestrator
- Compiles and returns the results to handler
- Routes the requests to the gateway clients using a mapper for each application
- It aggregates the responses received from multiple gateway clients
- Routes the requests to the DB clients using the mapper for each data store
- It aggregates the responses received from multiple DB clients
- Acts as a bridge between the uAct boundary and source application domain
- Maps the client response to uAct entity object
- Acts as a bridge between uAct boundary and uAct data stores
- Maps the relevant data store object to uAct entity object
- Handles interaction with a variety of uAct data stores, each catering to a different requirement:
- MySQL client manages user requests/approvals
- CacheFront client manages the user preferences
Currently uAct integrates with 10+ source applications; it is anticipated that with time this number will only increase many times over. Despite this, the application should remain highly performant, scalable, and with low latency for end users. uAct follows a hybrid architecture so that it can cater to different latencies and interfaces offered by the source applications.
If the source application provides a direct interface to pull data at the user level, and the API latency is within permissible limits, then uAct fetches requests at run time. To limit the latencies further, uAct parallelizes requests at the application level and also employs pagination. This way multiple pages of data are fetched at the same time. The getAction API requests fan-out at multiple levels with strict timeouts. This ensures degradation in any application does not impact the overall experience of users.
Each source application sends requests to uAct with its own set of fields. So, the main focus in design was to make the request structure consistent and uniform. To solve this problem, uAct only stores fields that are common across source applications, at the schema level. The remaining fields are in a variable-sized JSON object referred to as RequestMetadata. RequestMetadata is converted to a string for efficient storage, retrieval, and transfer.
The request schema:
uAct UI provides flexibility in parsing RequestMetaData. It uses a custom parser to parse application-specific information in the common request card component. As shown below in the first image, Product, Request Level, and Description are populated whereas in the second image Report ID, Last Modified, Last comment, and Country are populated in the same request card component.
Handling Diverse Applications
Each source application has its own architecture/framework/protocol. Below are some diverse patterns that uAct deals with:
- Database vs. API Interaction
- Queue-based event stream consumption vs. Bulk report (query builder) consumption
- Rest Https vs. Thrift vs. gRPC protocol
- Complete system dump vs. time-based delta dump
- On-demand Pull vs. Constant Polling
To handle this uAct uses Apache Camel framework, which implements Enterprise Integration Patterns and helps in easy integration with various applications. It provides a rich library of components to interact with other applications out of the box. It allows the creation of new custom components and their associated routes. The routes are invokable via HTTP endpoints as REST service. Apache Camel routes provide cron job schedules to handle long-running jobs.
Each uAct route does the following:
- Fetches chunks of data from source application
- Validates and ingests the data
- Maps application-specific data into uAct’s generic data schema
- Ingests data into the uAct data store
- Repeats all these steps until ingestion is complete
uAct provides personalization ensuring its UI performance does not degrade. In the future, uAct will support enabling/disabling the visibility of any source application as part of user preference. The getRequest API call has to be sequential to the personalization API call so it should be low latency.
uAct uses CacheFront to store user preferences. CacheFront is an Uber internal integrated caching solution. It is a transparent caching layer on top of Docstore, which eliminates the need to interact with both cache and storage. It provides low-latency reads and writes while abstracting the caching complexity.
Ease of Onboarding
In today’s highly changing tech landscape, new applications are added to the enterprise ecosystem quite frequently. uAct needs to be able to onboard these newer applications seamlessly with minimal or no changes to its existing schema and backend.
As per the design shown in Figure 3, uAct uses Handler, Controller, Orchestrator, Mapper, and Client layer in sequence.
uAct opted for a design where new applications do not interact at the handler or controller layer. Depending on the type of ingestion being used for a source application, uAct onboards via the following approaches:
- Real-time Ingestion source applications: For these applications, uAct only needs to add a new Gateway Mapper to interact with the source application client. The business logic of the Controller layer does not need any changes, hence does not hinder the existing functionality. This also means minimal development effort for the dev team.
- Offline Ingestion source applications: For these applications, requests are stored in MySQL DB. Common MySQL query is formed to fetch the requests on demand. In this case, no explicit changes are required on the backend.
Minimize Redirects from Platform
The aim of uAct is to minimize the number of clicks and redirects to different platforms. However, few applications do not allow third-party integrations with their application for actions due to technical and security constraints. For such applications, uAct leverages iFrames to support the actions.
iFrame stands for a nested window on the existing window as shown in the image above. iFrame loads the application’s request page without the need of redirecting the user to a new tab. Enabling iFrames is not straightforward as content security policy prohibits the rendering of external web pages on any website. Also, the uAct domain should be whitelisted in the frame-ancestors of the source application.
User Profile Enrichment
Email IDs are used as the unique identifiers of approvers and/or requestors of the requests in most applications. Users can make more informed decisions on the requests in uAct if they could see more details about the requestor, such as name, profile picture, link to Whober (Uber’s employee directory), etc. The main challenge with this requirement was to ensure users aren’t impacted in case of degradation or failure of profile enrichment API.
In order to solve this problem, uAct decouples the main user request from the profile enrichment flow. Initially, the UI loads the requests without the user profile information. The email IDs of all the employees involved in the requests are collected. The extra requester information is loaded lazily in the background, simultaneously for all requests on the page. Refer to the GIF shown below.
This way, the user can continue with other activities on the platform even while the profile details are loading. In case the call to fetch employees’ information fails, the UI does not break and shows only the employee’s email. Refer to the GIF below where the employee’s profile image is not rendered and the UI shows a default thumbnail in blue.
Trust on Platform
uAct serves as the centralized platform for actions from multiple applications and it contains personal data that should not be accessible to any other user. Hence it becomes vital to have authentication and authorization for handling user data.
The uAct UI application is managed via company-wide centralized authentication (Onelogin Auth) so only Uber employees can access it. uAct platform also provides an additional layer of authentication and authorization on the backend service. Any call to the backend service passes via Identity and Access Management (IAM) which ensures the user can access only authorized data. For debugging purposes, the admin role for the development team has been defined. All admin activities are getting traced in IAM.
It is crucial to maintain an audit history for all the actions taken within uAct by users. Actions could be pertaining to important HR requests, systems/application access, etc., which need to be audit logged for legal and compliance reasons. Users can also view all these actions in a dedicated view in the UI.
Additional Capability on Web Interface
Manager’s & Skip-Manager’s Views
A majority of pending requests are on direct managers or skip managers. uAct provides a ready filter to view requests from direct and roll-down reports (i.e., My report’s team: skip manager view). uAct maintains the same privacy and confidentiality as on the source applications. Hence managers and skip managers see only those requests that have them in the approval chain.
uAct UI provides the flexibility for users to create their own default view filtering source applications or requesters of their choice. The platform stores and renders a customized default view to each user.
Users can clean up the clutter in their view by hiding the requests they don’t want to see in uAct. This does not delete the request from uAct or the source application, but only hides it from the UI. The user can easily view such “hidden” requests in a separate view and can opt to “unhide” these when needed.
Now that we have successfully implemented uAct across Uber, achieving our goal of unifying our myriad internal notification systems into a single, convenient platform, here are some of our planned initiatives to upgrade it further:
Add New Source Applications
Leveraging the existing orchestration framework, newer source applications will be integrated with uAct, such as PagerDuty, Jira, GitHub, Coupa, Concur, DocuSign, Asana, ServiceNow, Charter policy, and many more.
uAct platform will integrate with Slack for notifications. Users will be notified over Slack whenever any new request is assigned to them. Additionally, users will be able to change their notification preferences such as frequency, include/ exclude notifications for certain source applications, etc.
uAct platform will email a consolidated daily or weekly digest/report of all their incoming requests.
The nudging feature will allow users to send reminders to approvers in order to expedite the action on the request. This will push the request to the top of the list of the approver, so they don’t miss it. In case there is no action on a pending request, requestors can escalate a request to the approvers’ manager.
With this feature, uAct will allow moderators to create company-wide actionable announcements such as taking annual company surveys, undertaking mandatory security-related tasks, etc. Moderators would be able to schedule posting these announcements, select target audience, and view analytics on it too (number of views/clicks).
Personal and Shareable To-Dos
uAct will have the capability to create both personal and shareable action items. Users can create action items for teammates and utilize uAct to track and follow up on such items.