NuGet: To Reference, or to NoReference? That is the question.

To Reference, or to NoReference, that is the question
Whether ’tis nobler in the mind to suffer
The slings and arrows of outrageous NuGet references
Or to take arms against a sea of DLLs

Since 2016, Sitecore has made platform DLLs available as NuGet packages on their MyGet feed. Consuming library files vis NuGet is generally regarded as an industry wide best practice, rather than checking the files into source code. It is even more useful for open source and publicly available projects since Sitecore licensing explicitly prohibits you from making them publicly available.

With References vs No References

There are two types of NuGet package available on the Sitecore MyGet feed: regular (With) Reference packages and a .NoReferences package. It has been generally recommended within the Sitecore Community that the NoReferences package should be used within your own projects, since this type of package will only add a reference to the single DLL you need and not bring in any of the dependencies.

When we asked via a poll on Twitter, a massive 89% of respondents said they used the NoReferences packages.

The stats on MyGet also highlight the love for the NoReference packages. For example, Sitecore.Kernel.NoReferences has been downloaded 140,769 compared to Sitecore.Kernnel (With References) package which has been downloaded 60,136 times, and even then, this number probably includes a fair few accidental install by developers choosing the first package in the search results, promptly followed by reverting that change and installing the NoReference package. I’ve done this a fair few times myself 😊

So what’s the big deal? Use whatever you want, makes no difference, the project still builds right? Take a look at the difference in project reference which adding the Sitecore.Kernel package:

The NoReferences package is must “cleaner”, we only have the single DLL added to our project reference, but the regular package has brought in an additional 30 package references 😱 In my opinion, the first option also much more clearly shows the intent of the developer as well.

When we check the list of NuGet package installed, we see a similar story as well:

We now have 33 packages listed, instead of that single Kernel that I actually wanted. This is also reflected in our packages.config, we have all packages listed as a dependency, as you would expect.

Why not just ignore the dependencies?

Due to these reasons, most of us have tended to use the .NoReferences packages, although from a Nuget(.org) perspective, nearly all packages only come in one flavour – with References. And this makes sense. If you were developing an application, why would you not want the dependencies, you need to build/deploy/run after all. But with Sitecore, we are developing on top of a framework, and we deploy our code on top of a base installation of Sitecore, so we already know that those dependencies are already on the server.

Having been fixated on No References, it seems that most of us missed the fact the you can install only a specific package and specify that dependences should be ignored. This gives us a similar behaviour to using NoReferences packages whilst following industry guidelines.

Adding Packages

When adding a Nuget packages from within Visual Studio, you can change the Dependency Behaviour option to Ignore Dependencies.

If you want to use the Package Manager Console, then pass in the -IgnoreDependencies parameter:

Install-Package -Id <package name> -IgnoreDependencies -ProjectName <project name>

You can read more in the NuGet documentation.

Updating Packages

When updating packages from within Visual Studio, ensure the Dependency Behaviour option is set to Ignore Dependencies.

To update the package using Package Manager Console, pass in the same parameter:

Update-Package -Id <package name> -IgnoreDependencies -ProjectName <project name>

Project References

Now when we add a reference to Sitecore.Kernel we have a clean project like we did we NoReferences packages:

Limitations

So far so good. But caution needs to be taken to pay attention:

When you set the Ignore Dependencies option in Visual Studio, you are toggling a UI switch only. This switch remains the same until you change it. If you switch it to install a Sitecore package and then try to install another package such as ASP.Net MVC where you do want the dependencies, then you’ll need to flip to back. If you then need to install a different Sitecore package, you’ll need to flip it back again, etc. Same goes for when you want to update that package. So pay attention when using this interface.

Unfortunately there is not way to set in config a specific source to always ignore dependencies. That would have made this approach great if it was possible.

PackageReferences

There is another, more modern option. This is likely the longer term solution to the issue, and for those who are deep in the .Net Core world this will be second nature to them, since this is the default approach.

PackageReferences is the successor to packages.json that tried to solve some of the mess that occurred package.config made to the .csproj when you installed a NuGet package, but it wasn’t without its own problems.

It’s worth reading about the Evolution of the NuGet PackageReference to understand a bit of the history and this post Kam Figy previously wrote about NuGet.

What are the benefits of using this format?

Some high level benefits of using PackageReference format over the previous ones:

  • Dependencies managed in a single location

    Package references are managed in a single location only. No more packages.config or .json file, the reference is now moved to the .csproj file directly, with just a single entry per package. For example:

<itemgroup> 
    <packagereference Include="Sitecore.Kernel" Version="12.0.0" /> 
</itemgroup>

That’s all. And it works the same across different project types since it is part of MSBuild.

  • See only the dependencies you care about

    Any references to packages you may add are super clean, since it uses Transitive Package Restore. Only the top level packages that you directly installed in the project reference is added and listed. This means both the Package Manger UI and project file are not cluttered with down-level dependencies, much like the NoReferences we are used to using and without having to set Ignore Dependencies with the regular packages.

    You can manage global packages and cache folder from your project nuget.config file should you need to change this behaviour on a per solution basis. I hope you are all using a solution specific nuget.config already…

  • Performance Improvements

    When you restore your NuGet with packages.config, the packages are downloaded to the /packages folder of your solution. When using PackageReference, a global-packages folder is used instead, which leads to performs faster and consumes less disk space since all projects will consume the same set of packages. This may be less noticeable if you only work on a single project, but if you frequently switch between projects this will become more noticeable. It will however be of most benefit on your build server, especially if you have many branches and/or projects actively being worked on, since they all share the packages from the global cache rather than having to download every single build.

    The way that packages are restored have also changed. When packages are restored, the dependency graph is calculated and flattened up front.

  • Control over dependencies and content flow

    Since NuGet is now fully integrated into MSBuild and a first class citizen, we get finer control of dependency reference. We can conditionally reference a NuGet package based on target framework, platform and various other pivots. You could use different build configurations to allow you to target different versions of Sitecore, for example.

    We can also take benefit of Floating Wildcard Versions, and since Sitecore itself is using this for their dependencies, we can upgrade our own usage of common packages (such as Newtonsoft.Json) to a higher release that OOTB Sitecore ships with.

  • PackageReference being actively developed

    This is the future direction that Microsoft envisages we should all be moving towards. As such, future development efforts are only being directed towards PackageReference enhancements, and packages.config is no longer under active development.

There are quite a few new features that can be utilised, I’ve added some links towards the end of the article that are worth reading for further details.

Set project to use PackageReference

In order to use this new format, you need to change your settings in Visual Studio. You can set it to always use a specific format, or to ask you when you create a new project, which is useful if you are working on multiple solutions and you cannot or do not want to switch them all over.

From the menu, select Tools > Options > NuGet Package Manager > Package Management:

Visual Studio Package Management

Read more here.

Now when we add out first NuGet reference we will get a prompt to select the Package Manager format:

Using Project References

We’ll create a new project and select to use Package Reference this time, and when we add a reference to Sitecore.Kernel package, with all the reference we get a super clean reference in Visual Studio 😍

Adding the package works in exactly the same way, and the same command if you want to use command line. The NuGet Package Manager also only has the single reference to the top level package that we added, which makes updating easy and the shows the developer intent more clearly, since we are just dealing with the specific package that was added.

Visual Studio Package Manager Format

You’ll notice the special icon for the referenced package that we added too.

Migration

In theory, migration is built into Visual Studio. But if you read the small print then you’ll see that this is not available for ASP.Net projects, probably due to the below issues. There are also a couple of Visual Studio plugins that will automate the migration, but this proved to be a little buggy in our testing, and worked with a limited number of projects.

A sample PowerShell script has been created by Nick Wesselman to help with migration, please read his article linked below though.

You will also need to be using Visual Studio 2017, this format is not supported by earlier versions.

If it’s so great, then why isn’t everyone using this?

Most likely compatibility. And possibly the lack of automated migration for ASP.Net projects may be due to existing packages, and their incompatibility with this new format. Plus there are a number of known “issues” using this format.

NuGet has been around and popular for a good number of years now, especially amongst ASP.Net projects. I find it hard to remember the time before NuGet was used. This has meant there are a huge number of libraries that are simply not compatible with the new format, and not just libraries used in Sitecore projects, a fair few purely .Net ones too.

  1. If a NuGet project added files to your project (config, xml, js, css, whatever) then most likely it was placed in the content folder, but PackageRefrence expects this in the contentFiles folder. You must manually copy/update the files into your project from the NuGet package.

  2. If a NuGet package ran a install.ps1 or uninstall.ps1 when it was added/removed, then PackageReference will not run this script and you must instead (remember to) manually take this action.

  3. If a NuGet package applied an XDT transform to files, then this will not be run either. You must again manually do this.

Within our Sitecore bubble alone: (1) affects packages like Glass Mapper, Unicorn and Dianoga which all include config files and other tool files, (2) affects Glass Mapper V4 which used install.ps1 to determine which version of the DLL to reference based on your own reference to Sitecore.Kernel and System.Web.Mvc, (3) affects Sitecore.Ship which applies an XDT transform on your web.config file. And these are purely a few examples quickly off the top of my head.

That’s not to dismiss using PackageReference format, just that some care and attention needs to be taken before heading down this path.

Sitecore 9.1

There are a number of changes around Nuget packages that have been introduced with the release of Sitecore 9.1.

  1. Sitecore is moving to Semantic Versioning finally! They will no longer use the product marketing revision number in the DLLs. This should also make updates easier to manage and plan, since you will know purely from the version number if it includes any breaking change (MAJOR), new functionality has been added in a backwards-compatible manner (MINOR) or it’s just to fix bugs (PATCH).

  2. A new Myget feed will be published for each major release of the platform. You will need to change the feed URL your project uses whenever you need to upgrade major versions, but hopefully that is as simple as changing a single line in your project-specific nuget.config

  3. NoReference packages will no longer be published. In line with the rest of the industry, only a single package will be published, and you must instead use one of the approaches outlined above.

  4. If you move using PackageReferences, you can take advantage of the floating wildcard versions. Sitecore will change all 3rd party references to use a range instead. This will allow you, for example, to update to a newer release of NewtonSoft.JSON within the same MINOR/PATCH release without causing build failures.

  5. A top level platform meta package will also be available on the feed, which will allow you to see the exact versions of a DLL/package that is contained in a specific release.

These are some interesting changes, and longer term the move to using proper semantic versioning can only be a good thing. For me, the high level marketing versions have been fine, but the revision numbers where not something that my brain would remember, I would always have to look back to figure out what update that was specifically. Following semantic versioning will also give businesses much higher levels of confidence on the amount of effort they are likely to encounter when upgrading, as well force the core product team think harder about the changes they will be releasing and how that affects the end customer upgrade path.

There is definitely some work on all our parts to move to this new way of working, but a positive step towards working with other industry practices.

Sitecore NuGet Feeds

For reference, the following are the URLs of the Sitecore MyGet feeds:

For all versions up to and including Sitecore 9.0:

NuGet V3 feed URL (Visual Studio 2015+)

NuGet V2 feed URL (Visual Studio 2012+)

Browse packages

For Sitecore 9.1, there is a version specific feed:

NuGet V3 feed URL (Visual Studio 2015+)

NuGet V2 feed URL (Visual Studio 2012+)

Browse packages

Further Reading

I highly recommend you read this post by Nick Wesselman on these changes, as well as a sample PowerShell script to help you migrate to the PackageReference format.

Additional Links