Paket is a popular alternative to the official NuGet client tool, which offers full use of the official NuGet package repositories whilst adding excellent value-add features. One of the areas where Paket has recently been lacking in comparison to the official NuGet client, however, is first-time performance i.e. When creating a new repository and adding a single package, how long does it take to add the package?

Paket by necessity has more to do than NuGet, because Paket by resolves all transitive (child) dependencies to the highest version possible. If you're a NuGet user, remember all those times you've installed a NuGet package and then immediately had to update the transitives? Paket does this automatically as a single step. But doing this is an expensive operation because it means Paket having to check NuGet to discover the newest possible version for every dependency when generating the lock file. Regardless, it's not always apparent to users as to just how much work Paket is doing - it simply appears to be a slow end-user experience.

We've been working on improving this over the past few weeks and, along with a few external factors as well (which I'll explain below), we've now gotten Paket first-run performance into a very good place (I say "we", but the majority of the credit must go to Steffen Forkmann for doing the bulk of the work on Paket).

1. Creating a benchmark

I'm going to start by creating a benchmark to measure against future improvements, using Paket 5.229.0 and creating a lock file for the popular ASP .NET compatible F# web library, Giraffe. Thanks to dotnet local tools, it's trivial to switch between different versions of Paket:

dotnet new tool-manifest
dotnet tool install Paket --version 5.229.0

First, I'm going to create an "empty" dependencies file

dotnet paket init

which generates the following empty default Paket Dependency file:

source https://www.nuget.org/api/v2

We can now add Giraffe to our dependencies by calling dotnet paket add Giraffe. This will calculate a transitive graph for Giraffe and all its dependencies. It's a pretty slow process, taking 49 seconds:

Performance:
 - Resolver: 30 seconds (1 runs)
    - Runtime: 918 milliseconds
    - Blocked (retrieving package details): 25 seconds (38 times)
    - Blocked (retrieving package versions): 4 seconds (5 times)
 - Disk IO: 13 seconds
 - Runtime: 49 seconds

To be fair, Paket is doing quite a lot, creating a fully up-to-date lock file 342 lines long. This is because Giraffe has 13 dependencies (such as various AspNetCore libraries and FSharp.Core), which themselves also have children. For example, one dependency in the lock file is System.Runtime.CompilerServices.Unsafe - and Paket can explain why this has been added:

dotnet paket why System.Runtime.CompilerServices.Unsafe

Paket version 5.229.0
NuGet System.Runtime.CompilerServices.Unsafe - 4.6 is a transitive dependency.
It is part of following dependency chains:

-> Giraffe - 4.0.1
  -> Utf8Json - 1.3.7
    -> System.Threading.Tasks.Extensions - 4.5.3
      -> System.Runtime.CompilerServices.Unsafe - 4.6

And so on.

2. Removing local packages

Notice above that around 25% of the time is taken up by Disk IO. That's because Paket has downloaded packages into the local folder by default - which takes up 326mb of disk space from NuGet packages (almost 3k files). We can fix this, and piggy back on the Nuget global package cache by adding a single line to the dependencies file:

source https://www.nuget.org/api/v2
storage: none

Deleting the lock file and re-adding Giraffe again now gives around a 40% performance improvement!

 - Resolver: 30 seconds (1 runs)
    - Runtime: 935 milliseconds
    - Blocked (retrieving package details): 25 seconds (38 times)
    - Blocked (retrieving package versions): 3 seconds (5 times)
 - Runtime: 31 seconds

Notice that Disk IO is now completely removed.

3. Switch to NuGet v3 API

The NuGet repository has two official APIs - the v2 and the newer, faster, v3 API. Paket supports both, but here has defaulted to the v2 API. Let's update our dependencies file to point to the new API:

source https://api.nuget.org/v3/index.json
storage: none

Re-adding Giraffe now gives the following results:

 Performance:
 - Resolver: 20 seconds (1 runs)
    - Runtime: 1 second
    - Blocked (retrieving package details): 15 seconds (49 times)
    - Blocked (retrieving package versions): 2 seconds (8 times)
 - Runtime: 21 seconds

That's another 30% speed up for very little work at all! However, we still wanted more - I personally would like to see Paket lock files generated on average in < 5 seconds.

4. Improving Paket's dependency resolution performance

Thanks to this release, Paket now will check the local package cache to find out about the list of transitives a package has rather than ask NuGet. Whilst this doesn't completely remove the need for Paket to communicate with NuGet during lock file creation (it still needs to ask for latest available versions of each transitive), this still improves matters. Let's update Paket to a newer version:

dotnet tool update paket
Tool paket was successfully updated from version '5.229.0' to version '5.236.0'.

Keeping the exact same dependencies file as above and re-adding Giraffe gives the following results:

Performance:
 - Resolver: 6 seconds (1 runs)
    - Runtime: 953 milliseconds
    - Blocked (retrieving package details): 2 seconds (88 times)
    - Blocked (retrieving package versions): 3 seconds (7 times)
 - Disk IO: 191 milliseconds
 - Runtime: 8 seconds

Not bad - that's reduced the total time to around 40% from the old version, down to 8 seconds! The resolver is much quicker now, with basically no network calls to retrieve package details.

5. Switching to netcore 3.0.

Up until now, Paket has been actually resolving more than one transitive graph - it's been doing it for every .net framework version supported by Giraffe (and its children). But if we know we're only working on netcore3.0, we can help Paket optimise itself for just that framework when calculating the lock file:

source https://api.nuget.org/v3/index.json
storage: none
framework: netcore3.0

Now adding Giraffe is even quicker:

Performance:
 - Resolver: 4 seconds (1 runs)
    - Runtime: 378 milliseconds
    - Blocked (retrieving package details): 2 seconds (38 times)
    - Blocked (retrieving package versions): 1 second (6 times)
 - Disk IO: 88 milliseconds
 - Runtime: 5 seconds

We've now hit that magic 5 second mark - checking the paket.lock file, you'll see it's now been simplified as a result of targetting just netcore3.0, clocking in at 172 lines (around 50% the original size).

6. Optimising packages for netcore3.0 and netstandard2.0.

With the move to netstandard2.0 and netcore3.0, package authors can now remove many dependencies from their nuget packages: not only the "netstandard" nuget packages, but also System packages such as ValueTuple as well as ASP .NET Core. As an experiment, we baked some local test versions of Giraffe and TaskBuilder.fs, to ensure that they targetted netstandard2.0 or netcore3.0, and removed any unnecessary dependencies from them. Then, I added them to Paket's source repository list:

source https://api.nuget.org/v3/index.json
source C:\Users\Isaac\Source\LocalPackages
storage: none
framework: netcore3.0

Here's the result:

 - Resolver: 2 seconds (1 runs)
    - Runtime: 197 milliseconds
    - Blocked (retrieving package details): 553 milliseconds (10 times)
    - Blocked (retrieving package versions): 1 second (4 times)
 - Disk IO: 58 milliseconds
 - Runtime: 3 seconds

Huzzah! Less than 5 seconds to resolve Giraffe, down from the original of 49 seconds. The transitive graph (and resulting lock file) is now much simpler - so much so that I can display it here in full:

STORAGE: NONE
RESTRICTION: == netcoreapp3.0
NUGET
  remote: https://api.nuget.org/v3/index.json
    FSharp.Core (4.7)
    Microsoft.IO.RecyclableMemoryStream (1.3)
    Newtonsoft.Json (12.0.3)
    System.Reflection.Emit (4.6)
    System.Reflection.Emit.Lightweight (4.6)
    System.Threading.Tasks.Extensions (4.5.3)
    System.ValueTuple (4.5)
    Utf8Json (1.3.7)
      System.Reflection.Emit (>= 4.3)
      System.Reflection.Emit.Lightweight (>= 4.3)
      System.Threading.Tasks.Extensions (>= 4.4)
      System.ValueTuple (>= 4.4)
  remote: C:\Users\Isaac\Source\LocalPackages
    Giraffe (4.1.0)
      FSharp.Core (>= 4.7)
      Microsoft.IO.RecyclableMemoryStream (>= 1.2.2)
      Newtonsoft.Json (>= 12.0.2)
      TaskBuilder.fs (>= 2.1)
      Utf8Json (>= 1.3.7)
    TaskBuilder.fs (2.2.0)
      FSharp.Core (>= 4.1.17)

7. Sensible defaults

Paket has now been updated to generate empty dependency files as follows:

source https://api.nuget.org/v3/index.json

storage: none
framework: netcore3.0, netstandard2.0, netstandard2.1

This fits much better with modern day .NET and, as seen above, will result in much quicker performance for the majority of your use cases.

Summary

This chart illustrates the overal perfomance gains seen:

And this graph succinctly shows how we've eliminated virtually all effort on Disk IO and package details by comparing "old" and "new" Paket, leaving just time for asking NuGet for version details and other miscellaneous computation effort.

A number of different factors have come into play to enable both this blog post and also to allow Paket to generate lock files much more quickly than before:

• Using dotnet local tools to allow rapid deployment of, and access to, Paket
• Using the NuGet package cache to reduce download times
• Using the latest NuGet API to improve network performance
• Using the NuGet package cache for supporting package resolution
• Taking advantage of netstandard2 and netcore3 for simplified package resolutions

What I'm especially happy with is that this is a great example of the benefits of working with the .NET tooling and ecosystem, rather than going it alone. There's definitely room in the .NET world for two package managers, but working with the net tooling platform has provided the foundational capabilities for much of the benefits above. Simultaneously, we shouldn't overlook some of the work done by individuals like Steffen for his consistent desire to improve Paket, or package authors like Dustin and Stuart who are working to keep Giraffe up-to-date with the latest changes to ASP Net Core.

Hope you have (fun _ -> Ok) with the new version of Paket and quicker resolution times!

Isaac