Re-implementing the Outbox Pattern in MongoDB

I have blogged previously about how we implemented the Outbox pattern on top of SQL Server and Websphere MQ to avoid distributed transactions/two-phase commits in an Event Sourcing microservice here. In that post, I explained that we use one transaction to persist the event and record an entry in an “Outbox” table to publish the event. 

I now have to implement the Outbox pattern again to solve the exact problem but this time on top of different technologies, namely: Solace and MongoDB. Solace is a message broker similar to Websphere MQ, and publishing to it does not require any changes to our previous implementation. However, MongoDB poses an issue as its transactions only support writing to a single document. So how can we ensure that writing the event and creating the outbox entry happens atomically?

The answer is: we cannot! Not while they are in separate documents anyway.

Therefore, we need to store the outbox information in the same document alongside the event. This is the only way we can guarantee atomicity. To do this, all our events are wrapped in a class that holds the event alongside the outbox information. 

public class EventData
{
    public DomainEvent Event {get; set;}
    public Outbox Outbox {get; set;}
}

public class Outbox
{
    public bool Processed {get; set;}
}

The domain is oblivious to the existence of the outbox information as this is not a domain concern but rather an infrastructure one. Hence, the repository is decorated with an Outbox repository that attaches this information before persisting the event.

Just like in the Outbox pattern I previously implemented, Hangfire is used to monitor the events collection for unprocessed outboxes via a recurring job. When an unprocessed outbox is found, the outbox repository returns the event data to Hangfire stripped of the outbox information. Once Hangfire publishes the event, this repository will mark it as processed.

This solution guarantees at-least-once delivery, but as I explained in my previous post, all downstream systems are idempotent to cater for this.