A pleasant walk through computing

Comment for me? Send an email. I might even update the post!

How (and why) to maintain a README.md file for internal projects: Onboarding and Continuous Integration

Contents

This won't be a "Complete and Authoritative 10 Steps to Perfect Readme Files." It's intended to be a good starting point.

What (the heck am I talking about)?

I'm talking about the README.md file that's typically at the root of Git repositories, especially when using GitHub, GitLab, BitBucket, and Azure DevOps.

Specifically, though, I'm talking about the Readme file that you use in your on-premise repositories. That is, your organization's internal projects.

Internal projects' Readme files are too often ignored. If they have a concrete and central purpose, they won't be.

Why (do I want one)?

The Readme file should be the starting point for your code's users. For open source, it starts with what the code is for and how to use it. It ends with information on how to build the code.

Internal projects have a different audience and different priorities. They are

  1. Help a new developer get the code working on her workstation.
  2. Be the source of truth for continuous integration (CI).
  3. Be the source of truth for the environment's version numbers.

Luckily these go hand-in-hand(-in-hand). The point of the build portion of CI is to prove the code can be built on a clean workstation.

But what does a "clean workstation" mean? Today, it could mean a fresh, default installation of Windows, Linux, or MacOS.

There are two parts to getting your code building and running on a clean workstation:

  1. Configure the Environment the code will build in. These are external dependencies.
  2. Get and Build the code along with its internal dependencies.

How (do I communicate the Environment)?

The Environment will be the operating system and global installations required by your application. In Azure DevOps Pipeline parlance, they're the build agent's capabilities. Here's an example for a solution that includes an Angular application and a .NET Core WebApi.

## Environment
> Changing the environment? Change the CI/CD pipeline!

Install                          | Version    
---------------------------------|------------
Node                             | 14.15.3
NPM                              | deps on Node
Angular CLI                      | 11.2.3
Azure Authentication             | latest
Visual Studio (recommended)      | VS 2019+

One way to think of this is: "If my repo has a build script, what do I not want it to manage?"

Your app's build script shouldn't try to install Node or Angular. It shouldn't even check if they exist. The first-use developer, or the person creating the CI pipeline, should refer to the Readme for environment details. The contract is that the environment will exist and the build script doesn't need to verify it.

It's easy, with Node apps, to know if it's an environment package. It gets installed globally using npm install -g

I recommend the table above being the Readme's only place version numbers exist for those dependencies. It's the source of truth. If you opt to include them in the Getting Started instructions, you need to remember to find/replace whenever the version numbers change.

Regardless, the Readme is the source of truth for environment version information.

How (do I communicate Getting Started)?

An excellent practice is to create a build script that will be used by the CI pipeline. This same script should be run before pushing the final commits for a pull request. This way, the CI process is being tested in advance.

At its very simplest, a .NET Core solution's build script could look like this:

dotnet build MySolution.sln -c Release -t:Rebuild
dotnet publish MySolution.sln -c Release --output "package"
cd my-angular-app
npm run my-angular-app:build:production

This rebuilds the solution and publishes the .NET assemblies to a "package" folder. The Angular app is built and published to its standard dist folder. In your own organization, the script may also run tests, recreate a database, etc. The point is, there's just one step to take.

From an onboarding viewpoint, this is great. The steps are basically "clone the repo, run the build script."

Our Getting Started section could look like this. Note that I'm recommending nvm or nvm-windows to the developer. This script also configures the Azure authentication to an Azure Artifacts package source.

## Getting Started
> Be sure to use the correct versions from above

**Environment**  
1.  Install PowerShell Core.
1.  Install NVM or NVM for Windows.
1.  Open a PowerShell *administrator* shell:
    ```powershell
    nvm install [version]
    nvm use [version]
    ```
1.  Open a PowerShell *user* shell
    ```powershell
    # Angular
    npm install -g @angular/cli@[version]
    # Azure Auth
    npm install -g vsts-npm-auth
    ```

**Application**  
    ```powershell
    # Clone source
    cd "C:\users\[user]\source\repos"
    git clone https://corporate/repo/path/MyApp.git
    cd MyApp
    # Set credentials for company packages server
    vsts-npm-auth -config MyApp/my-angular-app/.npmrc
    
    # Build for local development
    ./build.ps1
    ```

Again, I'm only indicating version numbers so that there's reduced chance of the version numbers in the Environment table not matching the version numbers in Getting Started. Of course, if you trust everyone to find/replace, there's nothing wrong with including them.

Why (is this valuable to CI/CD)?

The Getting Started steps are, basically, what the CI/CD pipeline should use, too. Here's what a simple Azure YAML file might look like for our solution. The steps are virtually identical.

trigger: main

pool: 
  vmImage: 'ubuntu-latest'
  
stages:
- stage: build_stage
  displayName: 'Build' 
  jobs:
  - job: 
    steps:
    # Environment capabilities
    - task: NodeTool@0
      displayName: 'Install Node.js'
      inputs:
        versionSpec: '14.15.3'
    - pwsh: 'npm install -g @angular/cli@11.0.7'
      displayName: 'Install Angular CLI'
    - task: npmAuthenticate@0
      displayName: Authenticate to Azure Artifacts NPM
      inputs:
        workingFile: $(System.DefaultWorkingDirectory)/my-angular-app/.npmrc
    # Build
    - pwsh: ./build.ps1
      displayName: 'Run build script'
    # Publish
    - publish: $(System.DefaultWorkingDirectory)/package
      artifact: package
      displayName: Publish

- stage: deploy_to_build_stage
  # etc

When (do I see an awesome template/example)?

Right now.

# My Awesome App

## Introduction
What the app is for.

## Environment
> Changing the environment? Change the CI/CD pipeline!

Install                          | Version    
---------------------------------|------------
.NET Framework                   | 4.8
PowerShell Core                  | 7.x
Visual Studio (recommended)      | VS 2019+

Sources:
*   [.NET Framework](https://dotnet.microsoft.com/download/dotnet-framework)
*   [PowerShell Core](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-windows)

## Getting Started
> Be sure to use the correct versions from above.

**Environment**  
1.  Install PowerShell Core [version].
1.  Install .NET Framework [version].

**Application** 
    ```powershell
    # Clone source
    cd "C:\users\[user]\source\repos"
    git clone [path-to-repo]
    cd [appname]
    # Build app
    ./build.ps1
    ```

## Daily Development
Things needed to smoothly develop, such as how to run service dependencies.

## When to run the build script
    ```powershell
    ./build.ps1
    ```
The build script simulates what the CI server will do, and is intended to catch build errors before they get to the server. The script should be run before the "final" PR commit. That is:

1.  Before pushing the final feature branch, AND/OR
1.  Before pushing to the mainline branch

## Troubleshooting
Things that might go wrong. These SHOULD end up on product backlogs and get fixed.

That's (a wrap!)

The key values of a Readme for internal projects are:

  1. Source of truth for environment dependency version numbers
  2. Rapid developer onboarding
  3. Matching steps used for building locally and in CI/CD

How can you keep this up-to-date? While I'm not hoping your turnover is high, whenever a new developer is brought on she should try to get the application working using just the documentation, and update if needed.

The proof that everyone's doing their jobs? The document won't need updating!

One Sheet Summary: Leadership is Language

I make these to post on my wall and help me learn the subject. Be sure to read the book!


Seriously, read Mr Marquet's entire excellent book. It's so packed that this summary doesn't do it justice.

Download the PDF Version

NuGet 5.7+ Ignores NuSpec CSProj Replacement Tokens - and other weird behaviors

Update
A stellar sleuth discovered that the error messages are related to whether the nuget.exe executable's "Unblock" attribute is checked. When it is (i.e. the file is unblocked), the error message doesn't occur. While it's still a bug, at least there's a solid workaround.

I submitted this issue in on 2020-08-28. It's had some comments and confirmations, but the moderators are having trouble reproducing it. I'm hoping this will help.

Source Code: ClassLibrary1.zip

Title

Here's the link to the issue.

NuGet again throwing exceptions "authors is required" "description is required", ignoring csproj/nuspec replacement tokens · Issue #9954

Environment

Windows 10 Pro
Visual Studio 2019
.NET Framework 4.8

More Info

There are multiple failures being reported below. The fundamental one is that NuGet 5.7 and above ignore token replacements in .nuspec files from .NET Framework project .csproj files.

Steps to Reproduce

Pre-setup

Check the Windows PATH environment variables (System Properties > Advanced > Environment Variables) at the user and system levels and be sure that no version of nuget.exe is on the path. One way to verify is to open a new command window and type nuget. If there's output, nuget is on the path.

A reboot may be required for the computer to recognize the path change. Normally, it only takes closing completely out of the System Properties editor.

Remember to open a new command window after PATH changes so the environment variable is reloaded.

Steps

  1. In Visual Studio, create a new .NET Framework Console Application.

  1. Open Properties\AssemblyInfo.cs add the values that are substituted for $description and $author:
    [assembly: AssemblyDescription("Desc")]
    [assembly: AssemblyConfiguration("")]
    [assembly: AssemblyCompany("Name")]
    
  2. Download NuGet 5.7 https://www.nuget.org/downloads
  3. Copy to the project folder
  4. Rename to nuget.exe.

It's important that the file be named nuget.exe.

  1. In the ClassLibrary1.csproj project folder, run

    nuget spec
    
  2. Reduce the resulting ClassLibrary1.nuspec file to the minimum required properties

    <?xml version="1.0" encoding="utf-8"?>
    <package >
      <metadata>
        <id>$id$</id>
        <version>$version$</version>
        <authors>$author$</authors>
        <description>$description$</description>
      </metadata>
    </package>
    
  3. Build the project or solution.

  4. In project folder, run nuget pack

Expected: Successful build of new package

Actual: Error that Author and Description are missing:

Attempting to build package from 'ClassLibrary1.csproj'.
MSBuild auto-detection: using msbuild version '16.8.2.56705' from 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin'.
Packing files from 'C:\Users\charl\source\repos\ClassLibrary1\ClassLibrary1\bin\Debug'.
Using 'ClassLibrary1.nuspec' for metadata.
Authors is required.
Description is required.

NuGet 5.6 Behavior

  1. In the above environment, replace the nuget.exe version 5.7 with version 5.6.

    It's important that the file be named nuget.exe.

  2. Run nuget pack

The pack succeeds.

Attempting to build package from 'ClassLibrary1.csproj'.
MSBuild auto-detection: using msbuild version '16.8.2.56705' from 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin'.
Packing files from 'C:\Users\charl\source\repos\ClassLibrary1\ClassLibrary1\bin\Debug'.
Using 'ClassLibrary1.nuspec' for metadata.
Successfully created package 'C:\Users\charl\source\repos\ClassLibrary1\ClassLibrary1\ClassLibrary1.1.0.0.nupkg'.
WARNING: NU5128: Some target frameworks declared in the dependencies group of the nuspec and the lib/ref folder do not have exact matches in the other location. Consult the list of actions below:
- Add a dependency group for .NETFramework4.8 to the nuspec

NuGet 5.8 and 5.9 preview Behaviors

Both versions exhibit the same bug as 5.7.

Behavior When Renaming nuget.exe

  1. Copy version 5.6 into the project folder
  2. Rename it nugetx.exe (It doesn't matter what it's renamed to, as long as it isn't nuget.exe)
  3. Run nugetx.exe pack

Unlike when named nuget.exe, version 5.6 pack fails with the unexpected error!

Attempting to build package from 'ClassLibrary1.csproj'.
MSBuild auto-detection: using msbuild version '16.8.2.56705' from 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin'.
Packing files from 'C:\Users\charl\source\repos\ClassLibrary1\ClassLibrary1\bin\Debug'.
Using 'ClassLibrary1.nuspec' for metadata.
Authors is required.
Description is required.

Behavior When Two NuGet files in Path

  1. Copy version 5.6 into the project folder and rename to nuget.exe
  2. Copy version 5.6 again into the project folder and rename to nugetx.exe
  3. Run nuget pack
  4. Run nugetx pack

In both cases, the pack succeeds.

Real World Concerns

Many organizations will, in a continuous deployment environment, use a known path to the latest NuGet package and expect it to be named nuget.exe. As seen above, those organizations will suddenly find themselves with failing package steps.

In order to continue, they must either stay on version 5.6 and accept the potential security issues there, or rework their pipeline.

   Older