Background Workers in F#

We have customers who recognise what a practical language F# is, using it to create internal applications supporting their business processes. We're very happy that they are as enthusiastic as we are about F#, and we're happy to help them maintain and improve their applications!

In one case, a customer with an on-premise suite of applications built using SAFE-Stack had some APIs to update a workflow and then send notification emails.

The scenario was that the API handler would directly send the email to an on-premise SMTP server. Our brief was to improve the resilience and reliability of the API. Our strategy to achieve this was to implement the Outbox pattern as simply as possible.

Keeping it simple

The change in the API handler was very straightforward: instead of constructing a MailMessage (from .NET's System.Net.Mail namespace) and calling SmtpClient.SendMailAsync, create a queueEmail function to write the email details to the database.

For the outbox processor we may have various approaches available depending on the existing application architecture. For example, in Azure an Azure Function or WebJob may be suitable. Our customer's on-premise applications are not being migrated to the cloud imminently, so a straightforward solution suitable for running on a Windows Server would be needed.

Ideally we would be able to use the same solution in development and testing on the Linux machines some of my colleagues and the customer's developers use. Can we do this in F#? Yes we can!

Practical F# solutions, powered by .NET

F# is not just a great language for writing mathematical or financial functions and business logic. Being a pragmatic language targeting .NET we also get access to the functionality .NET provides.

Out of the box, we get a "Worker Service" template with an F# version. We can create the project easily from the a shell prompt with the .NET SDK installed: dotnet new worker -lang F#. The project structure is clear and understandable.

Program.fs is where execution will start and initialises the service and its dependencies. We may not even need to modify this file:

namespace MyBackgroundService

open System
open System.Collections.Generic
open System.Linq
open System.Threading.Tasks
open Microsoft.Extensions.DependencyInjection
open Microsoft.Extensions.Hosting

module Program =

    [<EntryPoint>]
    let main args =
        let builder = Host.CreateApplicationBuilder(args)
        builder.Services.AddHostedService<Worker>() |> ignore

        builder.Build().Run()

        0 // exit code

Worker.fs is where we will do our processing. As this is a template, by default it does not do very much - printing the current time every second until the service is stopped:

namespace MyBackgroundService

open System
open System.Collections.Generic
open System.Linq
open System.Threading
open System.Threading.Tasks
open Microsoft.Extensions.Hosting
open Microsoft.Extensions.Logging

type Worker(logger: ILogger<Worker>) =
    inherit BackgroundService()

    override _.ExecuteAsync(ct: CancellationToken) =
        task {
            while not ct.IsCancellationRequested do
                logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now)
                do! Task.Delay(1000)
        }

Already I appreciated the simplicity of this approach when I recalled years ago creating Windows Services with .NET Framework as classes derived from `ServiceBase, overriding the OnStart and OnStop methods and having to create a ServiceInstaller` class.

For our specific scenario, I had migrated the SMTP code from the API handlers to a module within this background worker project, so all I still needed to do was replace the contents of the while loop with a call to the function in that module to perform the outbox processing:

while not ct.IsCancellationRequested do
    try
        logger.LogDebug("Checking for new emails at: {time}", DateTimeOffset.Now)

        // Create the database and email connections required by the outbox processor function
        // Configuration comes from an appsettings.json file
        use connection = new SqlConnection(configuration.GetConnectionString("DefaultConnection")) :> IDbConnection
        let smtpClient = new SmtpClient(
                configuration["Smtp:Host"],
                int (configuration["Smtp:Port"]))

        // Call the outbox processor function
        let! count = Email.processUnsentEmails logger connection smtpClient ct
        logger.LogInformation $"Processed {count} emails"

        // Wait for 30 seconds before checking again
        do! Task.Delay(TimeSpan.FromSeconds(30), ct)
    with
    | ex ->
        logger.LogError(ex, "Unhandled exception in email processing loop")
        // Still wait before trying again
        do! Task.Delay(TimeSpan.FromSeconds(30), ct)

F# and .NET - not just for Windows

We accomplished everything we set out to achieve for our customer: an implementation of the outbox pattern that has improved the resilience and reliability of our customer's APIs in an easily deployable service which can be run on Windows Server in production and also on Linux for development and testing - across several distributions used by us and our customer including Ubuntu and Arch.

Taking advantage of F#'s status as a .NET language gave us the best of a great language and a powerful framework and cross-platform runtime.