A pleasant walk through computing

Comment for me? Send an email. I might even update the post!

Scrum in Azure DevOps - A simpler (custom) process

Series Parts

Intro

The default Azure DevOps Scrum process is good, but I think it adds some unnecessary complication. Below I'll walk through inheriting from and modifying that process. I'm building on the work in my post A Simple Intro to Scrum in Azure DevOps, so you should start there if you need to.

Lots of what's below is my opinion, so take it for what it's worth. But don't discount it, that's mean.

The Framework

In my experience, when Developers, Scrum Masters, and Product Owners get their hands on electronic project management software, they add and add to what's already there. This is usually a mistake. Instead, pare down to the simplest thing that works, and only add features when needed.

As an aside, the other thing organizations do is try to enforce a single process on all teams. This is not Agile, because it removes team autonomy, self-organization, and effectiveness. The usual reason given is to make reporting across teams easier. That gives too much weight to upper management control. Instead, ask Developers to help figure out how to deliver cross-team reports. While you're at it, see if you can pare down those reports, too. Ask, "How is this report delivering user value?"

One way to find the simplest thing that works is imagine--or put--it on paper. Here's all we're really trying to capture.

The work item structure is:

Feature
|_Story
  |_Task
|_Bug

Stories and bugs move through a board.

A story or bug card--sticky note version!--might look like this and has all the relevent info plus pointers to more detail.

Try to keep your electronic version that simple.

A New Process

  1. Open Azure DevOps and open the Organization Settings > Boards > Process

  2. Select Scrum > Create inherited process, and name it SimpleScrum

  3. Open the SimpleScrum process. Under Work item types, we already have an essential set of types. Let's start by creating a work item type named Story instead of Product Backlog Item (PBI).

    No, you can't rename an inherited process work item type. I hate this, and so do you.

  4. Open the PBI work item type > States, and hide the Committed state.

  5. In the original tab, choose New work item type and name it Story

  6. Under Layout, add and update the fields to look like this. In one case, you're renaming Description to Value (orange), in others you're using an existing field (green), and in still others it's a new field (red).

  7. Under States, add two new states: Active in the In Progress category and Removed in the Removed category, and remove Committed.

    You can only remove an inherited state that's been hidden.

  8. Under Rules, create the following rule.

    • Name: Assign Active
    • When: A work item stage changes to . . . Active
    • Then: Use the current user to set the value of . . . Assigned To
  9. Return to the Work Item Types and disable Product Backlog Item

    Important
    Only disable PBI if you are not going to apply the new process to an existing project. If you are, leave PBI enabled so that you can change existing work items from PBI to Story.

  10. Open Bug and make these changes.

    1. Hide System Info, Acceptance Criteria, Priority, and Activity.
    2. Optional: I hide the Build fields
    3. Add More Info, Developer Notes, Testing Notes, Reported By, and Blocked

      Any fields added to Story will be in the "Use existing field" dropdown.

    Important
    Treat Bugs as a defects reported by production users. Why? Because it makes it much easier to track Bug fix cycle time. See below for tracking defects during development.

  11. Under States, add Active to In Progress, hide Approved and Committed

  12. Under Rules, create the same Assign Active rule as in Story

Pre-Release vs Post-Release Defects/Bugs

If your shop includes quality assurance, user-acceptance testing, or other non-unit testing, you should track your defects separately from Bugs. If those roles are in your team, then I like communicating those defects at the same level as Tasks. If they're outside the team, then I recommend tracking them at the same level as Bugs.

  1. Create a new work item type named PreRelease Defect with the same fields, states, and rule as Bug. Hide Description. Choose an icon you like.

OK, the work items are ready!

Backlog Levels

These changes make your new work item type availabe in the New item dropdowns on boards and backlogs.

  1. Open SimpleSrum > Backlog levels
  2. Edit Backlog items and add Story and PreRelease Defect. Set Default work item type to Story

Use the Process

You can create a new Project using the SimpleScrum process, but you can also apply the process to an existing project.

Warning
Applying a process does come with risks if there are non-matching fields. Be sure you read the documentation!

I'm going to apply the process to Baskets-R-Us (from Part 1 of the series) so you see what's involved and why starting with a new project might be better.

  1. Open All Process and click the Scurm Team Projects link

  2. For BRUW, select Change Process and choose SimpleScrum

Fix up the work items

  1. Open the BRUW project > Queries > New Query

  2. Return all work items

  3. Open Column Options and add State and Reason. Sort by Work Item Type and State

  4. Select all Product Backlog Items, right-click > Change Type, select Story.

  5. Select Commmitted items, right-click > Edit, change State to Active

  6. Select Approved items, right-click > Edit, change State to New

  7. Save items

    You may have other changes to make to your work items.

Fix up the Story Board

Story Board customizations are at the Team level

  1. Open the BRUW project > Boards. I was greeted with this message:

  2. Fine, I'll Correct this now. It's the board's column settings that need changing, which is what I want to do anyway!

  3. You can't delete any columns that have stories in them until you move the stories.

    • Keep all existing columns. Rename Ready for QA to Ready for Test, QA to Test. Map each column's state to Active for now. Save.
    • Clear all filters. Move stories out of Approved, Ready for Dev and Ready for Release
    • Remove columns Approved, Ready for Dev and Ready for Release.
  4. Open the board's Settings > Styles, add a styling rule

    • Name: Blocked
    • Card color: Red
    • Rule criteria: Blocked = Yes

If a Story's Blocked field is set to Yes, the card will show red.

Fix up the Task Board

Task Board customizations are at the Team level

  1. Open the Task Board (Sprints) Settings
  2. In Fields > Story, add a field to always show Parent. This lets you see a story's Feature if it has one.
  3. Add a Styles rule for Blocked as done above

What about work item templates?

Azure DevOps supports templates, and I wish they were useful. Maybe they will be for you, but what I want is to assign a default template to a work item for my team. For example, have a Bug template with starter info in Repro Steps that's automatically applied on any new Bug.

Until that happens, I'll leave you with Microsoft's documentation.

Use work item templates to update stories & more - Azure Boards | Microsoft Docs

Wrap Up

This post sets up a good, basic process. A key aspect is that work items at the same level are structured almost the same. This helps reduce cognitive friction switching from a Bug to a PreRelease Defect to a Story, and emphasizes that the primary difference between them is their name.

As one Scrum consultant I heard interviewed put it, "I think of every work item as a story."

In the next post, we'll put this process to work.

Git commit/checkout is a snapshot, not a delta

ERROR!
What I say below about how Git commits work, and the recommended pre-PR flow, is correct. However, what I say about pull requests is not correct. In the example below, Sabrina's PR would end up with Jasmine's change because a PR merges into master.

My own confusion came from a client situation where a branch was deployed to production, instead of being merged into master first. I didn't spot that detail right away in the pipeline's history.

For many of us with past source control experience (Subversion, Team Foundation Version Control, Visual Source Safe), there's a subtlety to a Git commit that can trip us up when it comes to build/deployment pipelines. We mistakenly think the pipeline will "get latest."

A Git commit is a snapshot of the entire source tree of the developer's workspace. It's not a record of only that commit's changes. This is why it's critical, before submitting a branch pull request or a commit to mainline (depending on your workflow) that you pull the remote repository's latest code into your local repository.

Imagine two developers, Jasmine and Sabrina. They're going to work on a repo with two files.

|_princess.txt
|_witch.txt

At 1pm, they both clone the remote, so both their local repositories exactly match. I'm showing the SHA hashes below.

HEAD aaaa111  <== this is the commit number
|_princess.txt  bbbb111
|_witch.txt     cccc111

Let's be really clear. Git doesn't know about the files. It knows about the hashes it uses to name its copies of the files. So Git's commit tree is:

aaaa111
|_bbbb111
|_cccc111

They each create a separate branch. At 1:30pm, Jasmine changes her file, commits, and creates a pull request. Her commit tree is:

HEAD aaaa222
|_princess.txt  bbbb222
|_witch.txt     cccc111

When the PR is approved and commit aaaa222 is deployed, those files are deployed.

At 2:00pm, Sabrina changes her file, commits, and creates a PR. Her commit tree is:

HEAD aaaa333
|_princess.txt  bbbb111
|_witch.txt     cccc222

This is the important part: Sabrina didn't pull from the remote, so her commit tree doesn't include Jasmine's changes.

When the PR is approved and commit aaaa333 is deployed, Git does not get the latest princess.txt nor should it. Git checks out the named commit tree, and that commit has the earlier princess.txt.

What Sabrina needed to do before the PR is:

git checkout main
git pull --rebase
git checkout myfeature
git rebase main
git push
# OR git push --force <== if the branch was previously pushed

If she does this, her commit tree for the PR will be:

HEAD aaaa444
|_princess.txt  bbbb222
|_witch.txt     cccc222

Now the PR will deploy the latest files because that's what's in commit aaaa444.

What do you mean by automated tests and continuous integration?

I had a client ask me this posts's title question recently. It's a good one. In this case, the person asking develops internal Python software, but that's not his primary job so he wasn't familiar with the deep world of enterprise-level development.

How do we explain this to a newcomer? Here's my answer, rooted in Azure DevOps. Perfect? No. Useful? I hope so!

Automated Testing

Any tests that don't require human interaction. These include:

  • Unit tests (testing discrete code elements such as a single function)
  • Integration tests (testing that modules all communicate with each other, external sites, back-end databases, etc)

Automated testing is essential, not optional. When practiced well, developers routinely have more test code than app code. Unit testing improves architecture by encouraging loose coupling. It also dramatically improves confidence when making changes.

Continuous Integration and Continuous Deployment

Typically, when there's team development, the "main" version of the source code is kept on a remote server. In our case, that's the Azure DevOps Git repositories.

That main branch is what's been deployed to production users, and must always be working. So, how do we ensure that our changes don't break the main branch? That is, how do we safely integrate our code to what's already there?

The two parts to the answer are branching and building.

  1. Develop new features in a short-lived branch (short-lived branches exist no more than a day or two)
  2. Pull the main branch code to your local workspace
  3. Rebase your feature code onto the main, so it's as if you just made your changes. You've integrated locally.
  4. Rebuild the application locally and run all tests.
  5. Push your feature branch to the remote Git server and create a pull request, saying "Hey, I want this branch merged into main!"
  6. Someone approves the PR, integrating into main.
  7. The main branch automatically builds and tests the code from scratch. This is key. It's as if a new developer is working on a new computer. We build/test the code once, then deploy it to different environments. What could go wrong?
    1. The code doesn't build.
    2. The tests fail
  8. If anything goes wrong, we're notified and the bad code doesn't get released to users.

It can be counter-intuitive, but the safest development method is to push changes frequently. Why?

  • Fewer changes are easier to fix
  • Rolling back a PR with only a few changes isn't a big deal
  • Integrating multiple times per day means the latest changes are working

When there is high automated test coverage and frequent integration, there can be enough confidence in the system to enable continuous deployment. That's where the code is automatically released to users as long as all environments build and test successfully. If there's a problem in a release deployment, it's easy to either roll that back or, better, rapidly fix the issue since there will have been a small number of changes.