Using the outbox pattern in SAFE Stack apps

With a naive approach to service interaction, it can be easy for your system to get into an inconsistent state. The outbox pattern provides a way to ensure consistency.

The problem

Imagine that an application is required to perform two actions together so that either neither or both actions are performed. This is difficult if each action can fail independently of the other: if the app tries to do one action then the other, it's possible for the second action to fail after the first action has succeeded.

As an example, there might be a requirement that, when a user registers with your application, their details should be saved into the database and an email sent confirming their registration. Without proper handling, failures could mean that an email is sent but the user is never registered in the database, or that the user is registered but is never sent an email. Neither situation is desirable.

The solution

The trick to the outbox pattern is that, instead of trying to perform a database change and another action in one go, an application uses database transactions to atomically make the database change and record the intention to perform another action. A separate monitor process is introduced to react to the recorded intention and actually take the action.

Typically a new outbox table is added to the database and the new process monitors that table. The word outbox is used because this pattern is typically used when sending messages via a message broker. In some cases it can make more sense to use something other than a new table to track the intention to take an action.

Multiple actions can be chained in this way, with the processing of each action involving an atomic database change including recording the intention to perform the next action.

A worked example

Overview

1. Your application receives a signup request.
2. In a transaction, a record is added to the user table and another record is added to the outbox table.
3. Later, the monitor sees the new record in the outbox table and:
   i. Sends the email.
   ii. Updates the outbox table to say that the message has been sent.

Note that because the email is sent before the outbox table is updated, it's possible for the monitor to fail to update the database. In that case the email may be sent multiple times. In this scenario that's deemed preferable to the user never receiving an email. In other scenarios you may find idempotence useful.

Implementation

A new outbox table is added:

CREATE TABLE EmailOutbox
(
    Id UNIQUEIDENTIFIER,
    FromAddress NVARCHAR(100),
    ToAddress NVARCHAR(100),
    Subject NVARCHAR(100),
    Content NVARCHAR(100),
    Sent BIT
)

A signup requests handler is added that adds the user and email outbox database records in a transaction.

    let handleSignup request =
        async {
            let user =
                { Id = Guid.NewGuid()
                  Username = request.Username
                  Email = request.Email }
            let email =
                { Id = Guid.NewGuid()
                  FromAddress = "cit@example.com"
                  ToAddress = request.EmailAddress
                  Subject = "Welcome!"
                  Content = "Welcome to the app!"
                  Sent = false }

             use transaction = ...
             do! addUserToDatabase user transaction
             do! addToEmailOutbox email transaction
        }

A job is added to the WebJobs-enabled SAFE app that checks for pending emails every minute. For each pending email, the job sends it and updates the database record to indicate that it has been sent.

type WebJobs () =
    member this.SendEmails ([<TimerTrigger "0 * * * * *">] _timer:TimerInfo) =
        async {
            let! pendingEmails = getPendingEmails ()

            for email in pendingEmails do
                do! Email.sendEmail email
                do! markSent email
        }

This is a minimal example to get the idea across. In a production application you'd likely want logging and exception handling. You may also want to use an Azure queue to queue email sending rather than sending the emails within this webjob.

Note that it's standard to see a generic outbox table rather than one designed with a specific action in mind (emails in this case). In this example, it seemed simpler to show a dedicated email outbox table, but if you're using a service like Apache Kafka to broadcast a lot of messages, you may want a single outbox table that can handle all sorts of different messages.

Extensions

• In this case, rather than a dedicated outbox table, it might make more sense to have an extra column on the user table that indicates whether an email has been sent. It could start 'Pending', and be set to 'Sent' once complete, with 'Queued' in between if you're using queuing.
• This could be extended with a step to register the user on Azure AD after adding a database record and before sending an email.

Further reading

David Leitner at SQUER has a blog post which gives a good overview of the pattern along with some alternatives.

Summary

When you have actions which must fail or succeed together, the outbox pattern can be used to ensure that your system is eventually consistent. This is achieved by attempting the actions one at a time, and attempting the next action until success has been confirmed. This can result in some actions being taken more than once. To mitigate this, design your actions so that either it's ok for the effect to happen multiple times or no effect occurs after the first execution (idempotence).