Using FSCheck to aid refactoring

In my last post, I illustrated how we can use FSCheck to prove specific cases around impure functions - something that was interesting, but perhaps not necessarily "real world". In this post I'd like to show you something that is a very real world use-case for property-based testing, and one that you can start benefitting from using today.

Refactoring code

One task that we often carry out as developers is refactoring code. This can be defined as follows:

The process of restructing existing code without changing its external behaviour.

It's well known in the software industry that we don't refactor code as often as we should. One reason that I often hear is that we don't have the "budget" for it, or that management won't "allow" it. However, there's another reason that most developers won't say aloud: we're often just too scared to do it! Of course, no developer in their right mind will dare admit this, but it's true - especially in codebases that have little to no test coverage, or are written in languages that provide a strong enough type system to naturally give you confidence in refactoring.

In fact, even codebases that have very good test coverage might not be capable of providing you with the confidence that you need, for several reasons:

• Manually-written unit tests may only cover certain cases
• It's not unusual when refactoring to change the signature of the function in some way - particularly if the method is side-effectful in some way.

FSCheck to the rescue

FSCheck provides a cheap way to prove that your refactoring hasn't broken anything. For many cases, the following generic property will do everything you need:

let areBothFunctionsTheSame original refactored args =
    original args = refactored args

That's it. This property is a function that takes in three arguments:

• original: The original function
• refactored: The newly-refactored function
• args: The common value to pass to both functions as input

Here's a simple example of two functions that have different implementations but the same behaviour:

let filterTryHead (items, predicate) =
    items
    |> Array.filter predicate
    |> Array.tryHead

let tryFind (items, predicate) =
    items
    |> Array.tryFind predicate

We can prove this using FSCheck and the areBothFunctionsTheSame property:

Check.Quick (areBothFunctionsTheSame filterTryHead tryFind)

We provide the first two values to the property, but not args. FSCheck will provide 100 different versions of this that match the right type for the functions provided, of gradually increasing complexity. In this case, our two functions are generic (this is a nifty feature in F# known as automatic generalization) so FSCheck will even provide input values for us in this case of different types such as int, char, string and bool!

A real world example

Let's take a domain from an e-commerce shop that wants to rate a given Customer. A Rater function takes in a Customer and gives back a Rating:

type Customer = { Purchases : int; Balance : int; Years : int }
type Rating = Good | Ok | Bad
type Rater = Customer -> Rating // here for illustrative purposes only - not required

Here's the implementation someone has written:

let rateCustomer customer =
    if customer.Balance < 0 then Bad
    elif customer.Purchases = 0 then Ok
    else Good

This is quite simple, but we're in F# so we can also use (and normally favour) pattern matching. In this (contrived) example, here's how someone might rewrite the above implementation with pattern matching:

let newRateCustomer customer =
    match customer with
    | { Purchases = 0 } -> Ok
    | _ when customer.Balance < 0 -> Bad
    | _ -> Good

At first glance this looks the same as the above, but there's one accidental change that the refactoring has introduced - the order of the first two checks in the function has been flipped. From this, can you tell for certain if this is a breaking change in behaviour - and could you give a simple example of it? FSCheck can:

> Check.Quick (areBothFunctionsTheSame rateCustomer newRateCustomer)

Falsifiable, after 10 tests (2 shrinks):

Original:
{ Purchases = 0
  Balance = -1
  Years = -2 }

Shrunk:
{ Purchases = 0
  Balance = -1
  Years = 0 }

Not only has FSCheck found an example of how there's a breaking change, but it's shrunk the test case to the simplest input possible: if a customer has 0 purchases and a balance of -1. In other words - when a customer is provided that satisfies both of these two rating checks simultaneously, the Balance check should always "win" - but the refactored implementation has inverted this prioritisation! Needless to say, flipping the order of the first two match checks fixes this.

This is, incidentally, also a great example of having business rules that aren't immediately obvious.

Conclusion

FSCheck can easily generate random inputs of growing complexity in order to test the outputs of functions, which is a great way to prove whether you have safely refactored your code or not. In my next post, I'll illustrate how we can safely refactor the sample "broken" refactoring to ensure that ordering of matches does not affect behaviour. See you then!

Isaac