A pleasant walk through computing

Comment for me? Send an email. I might even update the post!

TFS Continuous Integration Walk Through Part 1 - Installing TFS and Checking In a Test Project

ci-logoThis is part of a series of walk throughs exploring CI in TFS, starting from the ground up. The entire series and source code are maintained at this BitBucket repository.
https://bitbucket.org/bladewolf55/tfs-ci-samples

Bonus: Uninstall!

Here's how to uninstall TFS, if necessary.

  1. Open Windows Services.
  2. Stop the VSO Agent service(s).
  3. Copy the service name(s).
  4. Delete the service from admin command prompt: sc delete vsoagent.Nesbit.Agent-Nesbit
  5. Open SQL Management Studio or Visual Studio SQL Server Object Explorer.
  6. Delete the Tfs databases.
  7. Open Programs and Features.
  8. Select Team Foundation Server Express 2015, choose Change, then Uninstall.

Install

Requirements and compatibility

Note These instructions are for TFS 2015 Update 3

 

Note about Visual Studio and MS Build Technically, you don't need to install Visual Studio to build using TFS. But, practically, you do, certainly if you use the recommended Visual Studio task. If you want to stick with just MS Build, you might be able to get away with installing the  2015 Build Tools

 

This walkthrough installs TFS, a build agent, and Visual Studio on one machine to keep the demonstration easy.

  1. Download latest version (2015 Update 3 as of this writing). ISO

  2. If you're using SQL 2016, Download and install the C++ Runtime prerequisite. http://support.microsoft.com/kb/3138367

  3. Run installer
    2017-01-27_142237

    2017-01-27_142348

  4. After installation, the Configuration Center opens. Select Basic Server and click Start Wizard.

    2017-01-27_143404

  5. Choose whether to participate in the Visual Studio Experience Improvement Program and click Next.
    2017-01-27_143548

  6. Choose whether to install SQL Express or use an existing instance. In my case, I already have SQL 2016 Express installed. Click Next.
    2017-01-27_143631

  7. The default SQL instance is displayed, or enter a named instance. This is the instance in which the TFS databases will be created. Click Next.
    2017-01-27_143755

  8. The next screen is what threw me. "Install and configure a build and deployment agent" is unchecked by default. You can install an agent later (I show how), but it's just as easy to install one now. Check the box and accept the defaults. Note the installation folder for later use. Click Next.

    2017-01-27_144005
    2017-01-27_144159

  9. Click Next again at the review screen.
    2017-01-27_144341

  10. Readiness checks will run. If something fails, you'll need to fix it before continuing, then click "Click here to rerun Readiness Checks. Once everything passes, click Configure.
    2017-01-27_144511

  11. TFS and the agent will be configured. This creates databases, the agent service, and the website. When complete, click Next.
    2017-01-27_144541

  12. Note the web site location. It's a good idea to open the web site now. Click Close.
    2017-01-27_150639

  13. You're done with the wizard, so click Close. This will open the Server Administration Console.
    2017-01-27_150745

  14. Select Build and Release. This page has some instructions. You can go directly to the TFS Settings > Agent page by click on Download Build and Deployment Agent. (Note: We will not be using the legacy XAML Build Configuration.). You can also get to the Settings page via the gear icon on the TFS web page.

    2017-01-27_151017

    2017-01-27_151136
    2017-01-27_151306
    2017-01-27_151349

Console App with Tests

  1. Create a Console application
     2017-01-30_104608
  2. Add two Class Library projects to the solution. We'll show consuming different test frameworks.
    CIConsoleSample.MSTests
    CIConsoleSample.xUnitTests
     2017-01-30_105018
  3. In the MSTests class library, add an Extensions reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework
    2017-01-30_105737
    Also add a reference to the CIConsoleSample project.
     2017-01-30_111716
  4. In the xUnitTests class library, add the xUnit NuGet package, and the xunit.runner.visualstudio package.
    2017-01-30_113117
    And, as above, add the reference to the CIConsoleSample project.

Create MSUnit Tests

Rename Class1.cs to CIConsoleSampleMSTests, add the needed using statement and the following tests code. These tests use nested classes and a Class.Method_Should.Function naming style.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
//Added for tests
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace CIConsoleSample.MSTests
{
    [TestClass]
    public class CIConsoleSampleMSTests
    {
        [TestClass]
        public class TextMan_Greeter_Should {
            [TestMethod]
            public void AppendTextToTheGreeting()
            {
                //arrange
                string value = "Charles";
                string expectedResult = "Hello, Charles";
                TextMan textMan = new TextMan();
                //act
                string actualResult = textMan.Greeter(value);
                //assert
                Assert.AreEqual(expectedResult, actualResult);
            }
        }
    }
}

Create xUnit Tests

Reference: https://xunit.github.io/docs/getting-started-desktop.html
Add a class file named CIConsoleSamplexUnitTests.cs, then the following code. As above, these tests use nested classes and a Class.Method_Should.Function naming style.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
//added for tests
using Xunit;

namespace CIConsoleSample.xUnitTests
{
    public class CIConsoleSamplexUnitTests
    {
        public class TextMan_Greeter_Should
        {
            [Fact]
            public void AppendTextToTheGreeting()
            {
                //arrange
                string value = "Charles";
                string expectedResult = "Hello, Charles";
                TextMan textMan = new TextMan();
                //act
                string actualResult = textMan.Greeter(value);
                //assert
                Assert.Equal(expectedResult, actualResult);
            }
        }
    }
}

Run Failing Tests

First, here's how I like to display my tests, taking advantage of the nested class structure.

  • Tear off the Test Explorer into its own window, so that you can easily resize and see all the tests.
    test-tearoff
  • Group the test results by class
    test-byclass
  • In your xUnitTests project, add an app.config file with the following setting. This causes the result to display only the method name instead of the fully qualified name.
    <?xml version="1.0" encoding="utf-8" ?>
    <configuration>
      <appSettings>
        <add key="xunit.methodDisplay" value="method"/>
      </appSettings>
    </configuration>
  1. Build the solution so the tests are displayed in the runner.
     2017-01-30_120234
  2. Run the tests. They should both fail.
     2017-01-30_120353

This is enough for the moment. Let's add this solution to TFS, then to our CI server.

Add Solution to TFS

References
Create a team project Choose a process
Open the Team Explorer. If this is the first time connecting to TFS, you may see something like this (including the commercial).
2017-01-30_145139
Close the commercial, if you want, drop down Manage Connections and choose Connect to Team Project.
2017-01-30_145415
Select your local server. It will search for collections. Click Connect.
2017-01-30_145611
For this walk through, we're using TFS workspaces, not Git, for version control.
You can click "Configure your workspace" in the big mustard prompt. Or, click Project > Configure Workspace
2017-01-30_145755
The dialog maps the remote collection's root folder to a folder on your local computer. You can accept the default (which I'll do for this example), or choose another folder (which I'd normally do). Click "Map and Get". This is our first solution, so there are none to get.
2017-01-30_150019
Switch to Solution Explorer, right-click the solution, and choose Add Solution to Source Control.
2017-01-30_150759
I got this error message.

Full Disclosure I don't like TFS. This is an example of why. So much ceremony! I know there's a good reason, but sometimes I just want to get to work.

2017-01-30_150946
Switch back to Team Explorer, and choose Home > Projects and My Teams > New Team Project.
2017-01-30_153823
Name your team project. Click Next.
2017-01-30_153932
Choose the Agile process. Click Next.
2017-01-30_154431
Choose Team Foundation Version Control. Click Next, then Finish.
2017-01-30_154535
2017-01-30_154613
Now switch to Solution Explorer and add the solution to the team project.
2017-01-30_154745
The solution files are added locally, but not yet committed (checked in) to the repository. Right-click the solution and choose Check In.
2017-01-30_155043
Enter a comment and click Check In.
2017-01-30_155135
2017-01-30_155242
Next Part: TFS Continuous Integration Walk Through Part 2 - Create an Automated Build

What Else Programmers Do: A Text Manipulation Example

Sometimes, in my "regular" (i.e. non-IT) life, I apply my job skills. Today, for example, I came across a Facebook post about how to contact and influence Congress members. The post suggested cutting and pasting to share with others. Unfortunately, it wasn't possible to select the text. Some sites do this to prevent copying their content. News sites, especially, for copyright reasons.

What follows is not a tutorial for regular (i.e. non-technical-programming-nerdy) people. It's an example of the kind of process I do in my job every day, and how I think about a problem. It requires tools, knowledge and experience.

Getting to the "Source" of the Matter

Now, the first problem is that I viewed the original FB post on my phone, and it turns out that's why I couldn't copy/paste the text. If I'd bothered to find it on my desktop, my problem would have been solved. Instead, I emailed myself a link to the page. That link still went to the mobile version. That's what the "m." in the URL is telling me.

2017-01-26 09_35_13

Here's some of the text:

2017-01-26 09_38_16

I can't copy the text, it's sort of like it's an image, but I'm pretty sure it's not an image. My next step is to view the page source HTML. I right-click in Chrome and choose "View page source".

2017-01-26 09_40_46

Which yields a huge mess of coded text, as befitting a complex web site. There are about three thousand more lines like this.

2017-01-26 09_42_55

I'm pretty sure buried in there is the text I want. And I'm also pretty sure I'll need to manipulate it. So, I open Notepad++, a sophisticated, free, open source text editor. Then I copy/paste the source from the browser into it.

2017-01-26 09_46_56

Now I'm ready to use my text editor.

Text Editing is Not the Same as Word Processing

In word processing, like you do with Microsoft Word, the concern is text formatting: making things bold, italic, different fonts, etc. The result of all that work is marked up text (or binary, but we'll stick with text) that tells the application how to display all of your words on the screen. Here's an example from a Word file that was saved in its standard .docx format. The document literally has one word: "hello". But Word needs all of this information to display that those five characters.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<w:document 
xmlns:wpc="http://schemas.microsoft.com/office/word/2010/wordprocessingCanvas" 
xmlns:cx="http://schemas.microsoft.com/office/drawing/2014/chartex" 
xmlns:cx1="http://schemas.microsoft.com/office/drawing/2015/9/8/chartex" 
xmlns:cx2="http://schemas.microsoft.com/office/drawing/2015/10/21/chartex" 
xmlns:cx3="http://schemas.microsoft.com/office/drawing/2016/5/9/chartex" 
xmlns:cx4="http://schemas.microsoft.com/office/drawing/2016/5/10/chartex" 
xmlns:cx5="http://schemas.microsoft.com/office/drawing/2016/5/11/chartex" 
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 
xmlns:o="urn:schemas-microsoft-com:office:office" 
xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" 
xmlns:m="http://schemas.openxmlformats.org/officeDocument/2006/math" 
xmlns:v="urn:schemas-microsoft-com:vml" 
xmlns:wp14="http://schemas.microsoft.com/office/word/2010/wordprocessingDrawing" 
xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing" 
xmlns:w10="urn:schemas-microsoft-com:office:word" 
xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" 
xmlns:w14="http://schemas.microsoft.com/office/word/2010/wordml" 
xmlns:w15="http://schemas.microsoft.com/office/word/2012/wordml" 
xmlns:w16se="http://schemas.microsoft.com/office/word/2015/wordml/symex" 
xmlns:wpg="http://schemas.microsoft.com/office/word/2010/wordprocessingGroup" 
xmlns:wpi="http://schemas.microsoft.com/office/word/2010/wordprocessingInk" 
xmlns:wne="http://schemas.microsoft.com/office/word/2006/wordml" 
xmlns:wps="http://schemas.microsoft.com/office/word/2010/wordprocessingShape" mc:Ignorable="w14 w15 w16se wp14">
  <w:body>
    <w:p w:rsidR="000E45DD" w:rsidRDefault="00FA6736">
      <w:r>
        <w:t>hello</w:t>
      </w:r>
      <w:bookmarkStart w:id="0" w:name="_GoBack"/>
      <w:bookmarkEnd w:id="0"/>
    </w:p>
    <w:sectPr w:rsidR="000E45DD">
      <w:pgSz w:w="12240" w:h="15840"/>
      <w:pgMar w:top="1440" w:right="1440" w:bottom="1440" w:left="1440" w:header="720" w:footer="720" w:gutter="0"/>
      <w:cols w:space="720"/>
      <w:docGrid w:linePitch="360"/>
    </w:sectPr>
  </w:body>
</w:document>

The concern of text editing is working with raw content like that above. A good text editor lets you perform complex Find & Replace, or do block editing (where you can select text not just in rows, but also in a columns).

Back to our FB post source. I know that somewhere in the text is the bit that I want. So, I search for some text that's very likely unique to the content I'm looking for. "Senator" is a good choice.

2017-01-26 10_00_34

TECH DIGRESSION!

I found, at this point, why I couldn't copy/paste the text in the first place. All of the content was inside a <script> tag. It was probably being processed by javascript, and displayed in some messed-up way.

There's other markup in there, but I'll take care of that in a little bit. First, I copy what looks like the text want into a new Notepad++ tab.

2017-01-26 10_06_45

I Need a Replacement

Now, I look at what's there, while thinking about what I want. Ideally, my end result is nicely formatted text, meaning it's got all the original line breaks. I could do that manually, but I'd prefer not to.

The first thing I notice is the markup after "Senator."

From a high-level staffer for a Senator:\u003C\/p>\u003Cp> There are two things 

That sequence, \u003C\/p>\u003Cp> appears throughout the text exactly where line breaks are in the original.

2017-01-26 10_13_55

So let's see what happens if I just replace all of those instances with a new line and carriage return.

TECH DIGRESSION!

Huh? How (you wonder) do I do that? How do I insert something that can't be seen? Well, even though you can't see them, line breaks (where you go down one line) and carriage returns (where you go back to the beginning of the line) are stored in text just like any other character. They just aren't displayed in the editor. But there are ways to represent them, and one long-standing way is with C programming-style expressions. I can use text to represent other, hidden text.

 

\n = newline
\r = carriage return
\t = tab stop

 

The back slash is an "escape" character. It tells the application "don't treat this sequence \n literally. Instead, look in the raw text for the hexadecimal byte sequence 0D." As an example, here are two lines of text:

life's
good

and its hex representation. I put the represented letters and linefeed/carriage return above the hex for clarity.

l  i  f  e  '  s  \n \r g  o  o  d
6c 69 66 65 27 73 0d 0a 67 6f 6f 64  

So, I select \u003C\/p>\u003Cp> and try to replace it with \r\n\r\n (two carriage return + new line). And it fails.

2017-01-26 10_32_08

Why? For the same reason the newline can be found: the escape character \. Notepad++ is looking at my "Find what" string and saying, "Oh, you've got a back slash in front of that u. That must mean something special to me, so I won't treat it as '\u'. I can't find your text, sorry."

What do we do if we want the back slash to be treated literally? Heh, heh. You escape it by putting another back slash in front of it. To find a literal occurrence of \r in text using C-style expressions, you search for \\r. Crazy logical, right?

My search express becomes this: \\u003C\\/p>\\u003Cp>

I try my Find & Replace again, and this time it works.

2017-01-26 10_39_04

I could live with this result if I weren't a perfectionist. But there's still lots of HTML markup in there (highlighted in yellow). So, I do my Find & Replace magic a bunch more times. By looking at the original text, I can figure out what characters the markup stood for (or I could look it up). There's also some non-standard markup.

&#039;  =   '
&quot;  =   "
&amp;   =   &
&gt;    =   >
\u2013  =   --
\/      =   /

The result is the text as I originally saw it on the post. Here's a partial example.

From a high-level staffer for a Senator:

 There are two things that all Democrats should be doing all the time right now, and they're by far the 
 most important things. [***If you want to share this, please copy and paste so it goes beyond our mutual
 friends***]

 --> You should NOT be bothering with online petitions or emailing.

 1. The best thing you can do to be heard and get your congressperson to pay attention is to have 
face-to-face time - if they have townhalls, go to them. Go to their local offices. If you're in DC, try
 to find a way to go to an event of theirs. Go to the "mobile offices" that their staff hold periodically
 (all these times are located on each congressperson's website). 

 When you go, ask questions. A lot of them. And push for answers. The louder and more vocal and present
 you can be at those the better.

 2. But, those in-person events don't happen every day. So, the absolute most important thing that people
 should be doing every day is calling. 

 You should make 6 calls a day: 2 each (DC office and your local office) to your 2 Senators & your 1 
Representative.

Wrap Up

It took a lot to extract the text from that mobile web page.

  1. Use the browser's View Source feature
  2. Use a specialized text editor
  3. Find the content I was looking for
  4. Recreate the line endings via Find & Replace using C programming language expressions
  5. Replace the remaining HTML markup

If you're wondering what kind of things that special programmer in your life is doing every day, this is it. It's not glamorous. In fact, it's pretty tedious. But, it's a living!

What does this code do? Extended explanation

Ayende posted some code that didn't work as expected, and an explanation of why. You can read his full post here.

What does this code do?

But I'm not as bright as his readers, and was still mighty confused. It took me about ten minutes to finally see what was going on.

Ayende's original code is this:

var doc = new Dictionary<string,object>
{
    ["@metadata"] = new Dictionary<string, object>
    {
        ["@id"] = "users/1"
    }
    ["Name"] = "Oren"
};


Console.WriteLine(doc["Name"]);

The final statement fails, because doc is actually equal to a string value of "Oren", not the dictionary object. Why? And--for me--I couldn't understand why it was compiling at all.

It took me a while to see the syntax parsing, and remember that C# can be very forgiving of spaces. I'm not a beginner, but it's sort of a beginner mistake. For other readers like me who aren't as bright (if you have any!), here's another look.

While it appeared to me an element was being set completely outside of the declaration, the element is being set on the declared dictionary.

The final syntax is:

var item = NewDictionary[element] = value;

But that syntax can look like this:

var item = NewDictionary              [element] = value;

Code samples

//spaces before the element don't matter
var item = new Dictionary<string, object> {}    ["a"] = 1;
//is the same as
var item = new Dictionary<string, object> { }["a"] = 1;

//For clarity, we replace the declaration with its assignment.
var dic = new Dictionary<string, object> {};
var item = dic     ["a"] = 1;
//is the same as
var item = dic["a"] = 1;

I looked for other examples of where this can be done. The simplest array case doesn't compile:

//This is OK.
int[] items = new int[1] {1};
int item = items [0] = 2;

//But this isn't.
int[] items = new int[1] { 1 } [0] = 2;

Hashtable works the same as Dictionary, which is logical.

//OK
var h = new Hashtable();
h [0] = 1;

//OK, too.
var h = new Hashtable() [0] = 1;

Thanks, Ayende, for getting my brain working!