In the previous posts in this series, I discussed the current architecture, its limitations and a few alternatives. In this post, I will discuss a better option.
Atom Feeds
Let’s delve deeper into option two from the previous post in this series: Caching the event stream more efficiently. The goal is to keep a copy of the event stream in an accessible location that allows us to navigate it without affecting the service publishing the event stream. This is where Atom feeds come in.
Atom feeds are a popular way of disseminating event streams. They are low maintenance and leverage the infrastructure, so we don’t have to do any heavy lifting.
They remove the overhead of having to maintain local event streams. And more importantly, onboarding new services or aggregators has no bearing on the service producing the event stream.
Catchup Subscriptions with Atom Feeds
Atom feeds allow catchup subscriptions where the onus is on the consumers to track where they are on processing the event stream. This reduces the work needed to add a new consumer to zero on the producer’s side and localises the changes to the consumers only.
However, catchup subscriptions leave everything to the consumer. The producer publishes events to the Atom Feed, and that’s as far as its responsibilities go. Retries, competing consumers and everything else are left to the consumers to implement and manage.
Using Atom Feeds will simplify some issues at the expense of others that we get out-of-the-box with push notifications through, say, a traditional message broker. So why are Atom Feeds with catchup subscriptions the solution we wanted to use?
As with any architectural decision, it’s about trade-offs. There’s a requirement we value more than anything else when maintaining a reporting service or any read models: message ordering. Traditional message brokers pushing events out to consumers cannot guarantee ordering. However, you can define your consumption strategies based on each consumer’s needs with catchup subscriptions.
For this reporting service, we need to process messages in the correct order. Hence, we take control of the consumption and ensure that we only continue to the following message once the previous one has successfully been processed.
We use Polly to retry processing with exponential back-off to deal with most exceptions. Unlike retries in traditional messaging systems where messages are put back in a random position on the queue resulting in out-of-order processing, we only move forward when the message has successfully been processed.
Atom Feeds and Immutability
Since events are immutable, they are infinitely cacheable. This solves a couple of significant problems for us that I have already listed in previous posts in this series, namely:
- Adding new services
- Rebuilding existing reports/read models
Both scenarios would require consuming the entire event stream from start to end. As I explained in this post, that raises a few problems to avoid. At a high level, we need to send the entire event stream, but we also don’t want to keep doing this every time we need to rebuild or add a new service. Secondly, we need to publish and consume the event stream without negatively affecting the source service’s performance — read the linked post for a more detailed discussion. With the aforementioned Atom Feed, we solve both problems. We have the entire event stream cached in the infrastructure and accessible without resorting to the source system or affecting any other service interested in the same event stream.
In future posts in this series, I will discuss the limitations of Atom Feeds as well as the architecture in more detail and dissect our implementation and some of the trade-offs we made.
Originally published at https://medium.com/@ash-mageed on September 11, 2022.