Pages

Friday, March 3, 2017

TFS Continuous Integration and Private NuGet Package Sources

The Summary

What if you

  1. Don't store your NuGet packages in source control.
  2. Have a NuGet package that's hosted in a private source.
  3. Need to locally test continuous integration.
  4. Or need offsite (i.e. cloud) continuous integration.

There are a few general solutions.

  1. Store the private packages in source control. I'd do this one.
  2. Set up public access to the private source. Not likely.
  3. For testing, set up a local private NuGet source (just point to the folder).

The Problems

NuGet packages are generally great. But there can be problems when it comes to source control and continuous integration.

Most sites' advice on storing NuGet packages in source control (regardless of git, Mercurial, TFS, SVN, etc) is: don't.

The reason is pretty simple. If package restore is enabled, they'll get downloaded and rebuilt anyway, so why store them and take up repository space? With the later NuGet installations, a cache is maintained in the user's profile, so a trip to the NuGet servers might not even be necessary.

The counter argument is also pretty simple. What if the NuGet source isn't available? Suddenly, you can't restore your packages, can't build, can't work. When would this happen?

All of these assume you've downloaded from source control (such as GitHub or TFS), but haven't built the code yet.

  1. You get on a plane and then try to build. If you have no Internet, you can't get the packages.
  2. For some other reason you don't have an Internet connection.
  3. You're connected, but the NuGet site is down.
  4. One of the packages uses a locally (i.e. corporate internal) hosted NuGet source. It's not on the internet, so you can't download the package.
  5. It's been a long time since anyone's built the source, and a package has been removed from NuGet.
  6. You use the cloud for continuous integration, which won't have access to your private NuGet source.

Most of the above can be solved by building the solution immediately after getting it. But here's a real-world example of number 4. I was working for a client, and had access to their Team Foundation Server. I got the source via a VPN connection, copied it to my laptop, and tried to build. This client has several NuGet packages they host locally. They're proprietary, so hosting them on a public site like NuGet would be wrong.

And I didn't have access to that package source, so I couldn't build.

Again, these problems could be solved, and maybe they point to some environment changes needing to be made. But wouldn't it be just as easy to include the package in source control?

Including the NuGet Package Folder in TFS

If you're using a version of TFS prior to 2012, I can't help you (and you should upgrade). Starting in TFS 2012, the tfignore file became available. The purpose of tfignore is just like gitignore and hgignore: tell source control which files to not display for adding/tracking.

But it can also be used to explicitly allow files. Why is this needed? Because, unlike git or Mercurial, TFS + Visual Studio ignores certain files by default, and I haven't found a way to change that or even find out what files those are. Dlls are ignored by default, for example.

  1. Create the .tfignore file in the root of the team project.
  2. Edit the file.
  3. Add the package fies to TFS.
  4. Add your .tfignore file to source control.

The manual way to create a .tfignore file is, in the project folder root

  1. Right-click > New > Text File
  2. Enter .tfignore.

See the trailing period? That's the magic sauce. Otherwise, Windows doesn't let you create a file/folder with a leading period.

Visual Studio shows package folders in Excluded Changes by default. But dlls, and sometimes lib folders, are not included, and we absolutely need those.

Here's how to include the entire packages folder. The leading ! means "don't ignore".

!packages\*

These don't work. Note, especially, that leading backslash doesn't work, even thought that's what VS itself will create via the GUI.

  • !\packages*.*
  • !packages
  • !packages\

To include a specific package, such as one that's hosted in a private NuGet source, ignore the packages folder, then specify the package path without the version number.

packages
!packages\My.Private.Utility.*

In Visual Studio Team Explorer, choose Pending Changes, find Excluded Changes, click Detected: x adds(s).

2017-03-03_101910

Deselect all files (they're selected by default, a bad choice), then check the ones you want and Promote them (which is the same as "add" in git or Mercurial).

2017-03-03_103218

Using a Private Repo for Testing

If you're testing CI on a separate machine, you can use or modify a nuget.config file to include that source. I tried this at the project level, but it didn't work, so I created the nuget.config file at the solution level. (But that ability is supposedly removed in later NuGet versions. So confusing!)

Here's an example file:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageRestore>
    <add key="enabled" value="True" />
    <add key="automatic" value="True" />
  </packageRestore>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
  <add key="local" value="C:\Users\charles\Documents\Testing\NuGet\" />
  </packageSources>
</configuration>

Again, this must not end up in the production source control repository!

The Wrap Up

I'm not weighing in on storing NuGet packages in source control, except to say that there are some situations where it's clear to me it's a good ideal. Having a private NuGet source involved is one.

No comments:

Post a Comment