United Kingdom: +44 (0)208 088 8978

Keeping a database Docker image up to date

Matt talks through a FAKE script for keeping an application database Docker image, stored in Azure Container Registry, up to date with the latest database definition.

We're hiring Software Developers

Click here to find out more

Recently, I've been working on a project for which I made a Docker image containing the application's database, seeded with a minimal set of data to be able to develop with. This has been great! Whenever I want to start from a fresh dataset I can simply discard the database container that I'd been running and start a new one. I can have a clean database in a matter of seconds.

Even better, we store the database image in Azure Container Registry, which means that it's easy for other developers to pull the image and start developing.

As you might imagine, the database schema has evolved while we've worked on the project. This means that, in order to be able to run it and start developing straight away, the minimal database Docker image needs to be updated. Below is a script to pull the image from the registry, update the database and push the image back up to the registry.

#r "nuget: Fake.Core.Target"
#r "nuget: Fake.IO.FileSystem"
#r "nuget: Fake.Sql.SqlPackage"
#r "nuget: Fake.Tools.Git"

open Fake.Core
open Fake.IO
open Fake.Sql
open Fake.Tools
open System

let shellCommand executableOnPath args workingDir =
    let commandPath =
        ProcessUtils.tryFindFileOnPath executableOnPath
        |> Option.defaultWith (fun () -> failwith $"{executableOnPath} was not found in path")

    Command.RawCommand (commandPath, Arguments.OfArgs args)
    |> CreateProcess.fromCommand
    |> CreateProcess.withWorkingDirectory workingDir
    |> CreateProcess.ensureExitCode
    |> Proc.run
    |> ignore

let az args = shellCommand "az" args "."
let docker args = shellCommand "docker" args "."

// Pulls latest DB image from container registry, deploys latest DB changes and pushes resulting image.
// Must be authenticated with Azure for this to work. Run `az login` if necessary.
Target.create "PullDeployAndPushDbImage" (fun _ ->
    // Authenticate with container registry.
    az [ "acr"; "login"; "--name"; "registryname" ]

    // Run latest image from container registry.
    docker [
        "run"
        "--name"; "latestprojectdb"
        "--pull"; "always"
        "--env"; "'ACCEPT_EULA=Y'"
        "--env"; "'SA_PASSWORD=yourStrong(!)Password'"
        "--publish"; "1433:1433"
        "--detach"
        "registryname.azurecr.io/db/minimal:latest"
    ]

    // Wait for DB to start (hack - there's probably a better way to do this!).
    System.Threading.Thread.Sleep (TimeSpan.FromSeconds 10.)

    // Deploy DB.
    SqlPackage.deployDb (fun args ->
        { args with
            Source = Path.getFullName "./deploy/db/Database.dacpac"
            Profile = Path.getFullName "./src/Database/LocalDatabase.publish.xml"
            RecreateDb = Some false
            BlockOnPossibleDataLoss = Some false
            DropObjectsNotInSource = Some true
            Destination = "Data Source=localhost;Database=Project;User ID=sa;Password=yourStrong(!)Password" })

    // Create new image.
    let timestamp = DateTime.UtcNow.ToString("yyyyMMddHHmmss")
    let commitHash = Git.Information.getCurrentSHA1 "."
    let imageName = $"projectdb:{timestamp}-{commitHash}"
    docker [ "commit"; "latestprojectdb"; imageName ]

    // Stop container.
    docker [ "stop"; "latestprojectdb" ]

    // Add new tags to image and push.
    let specificTag = $"registryname.azurecr.io/db/minimal:{timestamp}-{commitHash}"
    docker [ "image"; "tag"; imageName; specificTag ]
    docker [ "image"; "push"; specificTag ]

    let latestTag = "registryname.azurecr.io/db/minimal:latest"
    docker [ "image"; "tag"; imageName; latestTag ]
    docker [ "image"; "push"; latestTag ]

    // Remove container.
    docker [ "rm"; "latestprojectdb" ]
)

You can integrate something like this into the build script/project for your SAFE app.

Let's examine the pieces one by one. I'll skip the shellCommand function so that we can focus on the Docker- and Container Registry-specific parts. It's sufficient to know that az [ "a"; "b"; "c" ] translates to az a b c in your shell (and similarly for the docker function).

Authenticate with the registry

az acr login... makes sure that the process executing the script is authenticated with Azure Container Registry. You may be prompted for credentials, and you will need to run az login first if you haven't already hooked up you Azure CLI to your Azure subscription.

Run the latest image

    docker [
        "run"
        "--name"; "latestprojectdb"
        "--pull"; "always"
        "--env"; "'ACCEPT_EULA=Y'"
        "--env"; "'SA_PASSWORD=yourStrong(!)Password'"
        "--publish"; "1433:1433"
        "--detach"
        "registryname.azurecr.io/db/minimal:latest"
    ]

This runs the db/minimal:latest image from your registry. --pull always ensures that you always run the latest version, --publish 1433:1433 makes it possible to connect to the DB running in the container from localhost:1433 on the host, --env SA_PASSWORD=... sets the password for the sa database user and --detach runs the container as a background process.

Deploy DB changes

SqlPackage.deployDb (fun args ->... uses the sqlpackage utility to apply changes using the dacpac file at ./deploy/db/Database.dacpac and publish profile at ./src/Database/LocalDatabase.publish.xml. How dacpac and sqlpackage works is beyond the scope of this blog post, but it's useful to know that you can use a database project to generate the dacpac. The other settings make sure that it's an incremental build (rather than dropping and recreating the database), allow the changes to be applied even if sqlpackage can't guarantee that no data will be lost and drop objects that have been removed from the dacpac definition. Note that the password we defined above is used in the connection string.

Docker commit, tag and push

  • docker commit snapshots the container after the changes have been applied (using the name we defined above) resulting in a new image with the name we pass as an argument (including the timestamp and commit hash)
  • docker image tag adds a tag to the image we just created
  • docker image push pushes (the image with) the new tag to the container registry

There's also a docker stop and docker rm to clean up the container once we're done.

Summary

I love having a developer database in a Docker image in Azure Container Registry, and a scripted way to keep it up to date with the latest database definition. Making it a target in the FAKE build script/project is even better.

I hope that you find this sample useful!