Skip to main content
Engineering

Building Reliable Reprocessing and Dead Letter Queues with Apache Kafka

February 16, 2018 / Global
Featured image for Building Reliable Reprocessing and Dead Letter Queues with Apache Kafka
Figure 1: When a pre-order request is received, Shop Service publishes a PreOrder message containing relevant data about the request. From there, each of the two sets of listeners reads the produced event to execute its own business logic and call its corresponding service.
Figure 2: If there is a breaking change in the downstream Payment Service, for instance, unexpected charge denial for previously valid pre-orders, then these messages would fail all retries. The consumer that received that specific message does not commit the message’s offset, meaning that this message would be consumed again and again at the expense of new messages that are arriving in the channel and now must wait to be read.
Figure 3: msg_a is an example message whose consumer receives an error response when handling it. This consumer publishes msg_a to the payment_retry queue and then commits msg_a’s offset relative to pre-orders, the original processing topic. With the consumption of msg_a complete, the consumer moves onto the next message.
Figure 4: Errors trickle down levels of retry topics until landing in the DLQ.

Posted by Ning Xia

Category: