A pleasant walk through computing

Comment for me? Send an email. I might even update the post!

Raw Code - Picture Renaming and Sorting

A while back I needed to rename a bunch of pictures, prefixing them with numbers, so that they'd sort the way I wanted.

Here's the code I used. I ran this in LINQPad, but you could use it in a console app with minor modifications. One nice thing I learned to do: get image datetime without loading the whole image.

Hope this is useful to you in some way!

void Main()
{
	string path = @"C:\Users\charles\Pictures\Inbox\20160630";
	List<FileChange> list = new List<UserQuery.FileChange>();

    foreach (string fileName in Directory.GetFiles(path))
	{
		DateTime taken;
		try
		{
			taken = GetDateTakenFromImage(fileName);
		}
		catch
		{
			try
			{
				taken = GetDateTakenFromFilename(fileName);
			}
			catch
			{
				taken = File.GetLastWriteTime(fileName);
				if (File.GetCreationTime(fileName) < taken) { taken = File.GetCreationTime(fileName); }
			}
		}
		FileChange item = new FileChange(fileName, "", taken);
		//set new taken on to old.
		item.NewTakenOn = item.OldTakenOn;
		list.Add(item);
	}

	//sort and set new name
	int newNbr = 1;
	foreach (var item in list.OrderBy(l => l.OldTakenOn))
	{
		item.DestinationFileName = GetNewFileName(item.SourceFileName, newNbr);
		newNbr++;
	}
	
	DumpList(list.OrderBy(l => l.OldTakenOn).ToList());
	DumpList(list.OrderBy(l => l.NewTakenOn).ToList());
	
	//rename
	list.ForEach(a => File.Move(a.SourceFileName, a.DestinationFileName));
}

public static void DumpList(List<FileChange> list)
{
	var newList = list
	.Select(l => new { oldName = Path.GetFileName(l.SourceFileName), newName = Path.GetFileName(l.DestinationFileName), l.OldTakenOn, l.NewTakenOn });
	newList.Dump();
}

//http://stackoverflow.com/questions/180030/how-can-i-find-out-when-a-picture-was-actually-taken-in-c-sharp-running-on-vista
//we init this once so that if the function is repeatedly called
//it isn't stressing the garbage man
private static Regex r = new Regex(":");

//retrieves the datetime WITHOUT loading the whole image
public static DateTime GetDateTakenFromImage(string path)
{
	using (FileStream fs = new FileStream(path, FileMode.Open, FileAccess.Read))
	using (Image myImage = Image.FromStream(fs, false, false))
	{
		//if (!myImage.PropertyIdList.Any(a => a == 36867)){ return DateTime.Parse("1/1/1900");}
		PropertyItem propItem = myImage.GetPropertyItem(36867);
		string dateTaken = r.Replace(Encoding.UTF8.GetString(propItem.Value), "-", 2);
		//string t = Encoding.UTF8.GetString(propItem.Value);
		return DateTime.Parse(dateTaken);
	}
}

public static DateTime GetDateTakenFromFilename(string path)
{
	var takenPartSplit = path.Substring(3, 15).Split('_');
	string datePart = takenPartSplit[0];
	string timePart = takenPartSplit[1];
	datePart = datePart.Substring(0, 4) + "-" + datePart.Substring(4, 2) + "-" + datePart.Substring(6, 2);
	timePart = timePart.Substring(0, 2) + ":" + timePart.Substring(2, 2) + ":" + timePart.Substring(4, 2);
	string datetime = datePart + " " + timePart;
	return DateTime.Parse(datetime);
}

public static string GetNewFileName(string path, int newNumber)
{
	string oldName = Path.GetFileName(path);
	string newName = oldName;
	newName = newNumber.ToString().PadLeft(3, '0') + " " + newName;
	return path.Replace(oldName, newName);
}

public class FileChange
{
	public string SourceFileName { get; set; }
	public string DestinationFileName { get; set; }
	public DateTime OldTakenOn { get; set; }
	public DateTime NewTakenOn { get; set; }

	public FileChange(string source, string destination, DateTime takenOn)
	{
		SourceFileName = source;
		DestinationFileName = destination;
		OldTakenOn = takenOn;
	}
}

Becoming a Professional Blogger Part 1

After watching this TED Talk by Josh Kaufman, I decided to take on a 20-hour challenge: to (start to) become a professional blogger.

This is part 1 of two entries. I'll follow up after July 1 to see whether I met my general goals.

The big reason I'm writing this post is...

After posting, I'll have reached my twenty hours!

Note: There are references below to a static site generator to be named. Stay tuned...I hope to release it in the next month or so.*

Define Success

My initial thoughts:

  • Publishing articles five days a week
  • Articles referenced by others
  • Writing articles for other sites
  • Ideally, earning some money

After research:

  • The classic definition is just like music: professionals get paid

By July 1, write 60 blog posts and have 5K unique pageviews per day

Resources

Definitions of professional blogger

Measuring Traffic

How To

SEO

PLATFORM

  • [Site-Generator-To-Be-Announced] (or other?)
  • Comments?
  • Measure traffic, daily and per-post. Need to know which posts are most popular.

Plan

What do I really like?

  • Methods, especially for self-improvement.
  • Certain tools, like Git.
  • Systematizing

Add Google Analytics to blog

  • Page views by date
  • Page views by post

Finish [Site-Generator-To-Be-Announced] enough to run as separate .exe
This is the first step to make the posting experience simpler. What I eventually really want is a UI that lets me:

  1. Quickly start a post for either Software Meadows or FlattLand
  2. Quickly continue a post
  3. Easily publish a post

Make [Site-Generator-To-Be-Announced] template(s) mobile-ready
Most people read on their phones. This is a must.

Gather blog post ideas and keep them on desktop
So that they're readily available, and easy to add to. Also use Keep for ideas, and consolidate weekly.

Write shorter posts
It's OK to write posts that are a few paragraphs. In fact, more people will read them.

Journal

2018-03-22 11:41:19

Gathered some resources, started trying to define success and pare down my goals.

2018-03-26 14:50:55

More planning, including tasks of adding analytics, and what needs to improve in [Site-Generator-To-Be-Announced] and the web sites.

2018-03-27 14:10:18

I've pulled my backlog blog ideas. Now off to Keith's to review/organize.

2018-03-27 16:30:38

Finished blog post about image resizing to target dimensions.

2018-04-04 16:00:00 Publish FlattLand post "Clean Keep using labels+archives"

2018-04-14 14:44:26

I needed [Site-Generator-To-Be-Announced] to be a single Exe. FodyWeavers (strange name) was the solution. I'm not even sure what this does, but there's a extension called Costura that packages up all the assemblies into a single .exe. Nice and easy!

https://github.com/Fody/Fody/

https://www.nuget.org/packages?q=Costura.Fody

2018-04-14 15:44:06

[Site-Generator-To-Be-Announced] command line is working fine. I wanted to use the Markdown Monster scripting addin to automate managing posts (at least temporarily), but no good. I suppose I could use LINQPad....

2018-04-15 13:03:26

I have Markdown Monster set up using the C# addin to quickly create and publish blog posts. It's a good interim solution!

2018-04-20 20:00:00 Begin time boxing post

2018-04-21 10:38:05

It's been a busy morning. I posted about my current time-boxing approach. Lots of research!

2018-04-21 21:22:46

Adding Google Analytics to a website? Easier than pie. Ten minutes from starting research to completion.

2018-04-27 15:33:54

About two hours of research into how to delete child entities in EF.

Writing a blog article now.

2018-04-27 18:58:47

Article is done. This weekend is about making the site mobile-friendly.

2018-04-28 06:22:09

Thank you, w3schools.com! You always seem to have the best simple introductions to a subject.

2018-04-28 08:13:13

It may not be the best, but it didn't take long to get my sites working for mobile (at least on my phone).

2018-04-28 10:10:47

Reviewing my ProBlogger progress. I'm about ten minutes from my 20 hours.

2018-04-28 10:31:08

Post part 1 of becoming a ProBlogger.

Entity Framework 6 Child Deletion and Foreign Keys - String Int GUID

The Problem

Here's a typical pattern for deleting a child entity we've all tried when using EF6.

public void DeleteItem(int orderId, int itemId)
{
  var db = MyDbContext();
  var order = db.Orders.Find(orderId);
  order.Items.Remove(order.Items.Find(itemId));
  db.SaveChanges();
}

Then we get one of these errors:

Cannot insert the value NULL into column 'OrderId', table 'MyDbContext.dbo.Items'; column does not allow nulls. UPDATE fails.

Or

The operation failed: The relationship could not be changed because one or more of the foreign-key properties is non-nullable. When a change is made to a relationship, the related foreign-key property is set to a null value. If the foreign-key does not support null values, a new relationship must be defined, the foreign-key property must be assigned another non-null value, or the unrelated object must be deleted.

This article explains why this happens and what to do about it. It also demonstrates configuring for five different key datatypes:

  • String
  • Manually-generated Int
  • Identity Int
  • Manually-generated GUID
  • "Identity" GUID

It's nigh-on impossible to find any EF examples of foreign key relationships that don't use a integer identity column for the primary key. Also, I haven't seen any author demonstrate the approach I'm taking, even though it's perfectly legitimate from an Entity Framework perspective. Simply put:

We don't have to model the database exactly as it is.

The Solution - Short Version for the Impatient

Most SQL databases I've seen create a one-to-many relationship this way. All of these columns are not nullable.

Order
-----
OrderId PK

Item
----
ItemId  PK
OrderId FK

This is technically called a non-identifying relationship. An item belongs to an order, but it's only required because OrderId is not null. OrderId isn't part of the item's primary key. The OrderId foreign key can change. The foreign key could be changed to nullable, and there could then be items that exist without an order.

Here's an identifying relationship. And item's uniqueness is bound to an order. The foreign key is also part of the primary key (it's a composite key). Thus, the foreign key can't change and the item must be part of an order.

Order
-----
OrderId PK

Item
----
ItemId  PK
OrderId PK, FK

Entity Framework will automatically delete child entities that are part of an identifying relationship. If the relationshiop is non-identifying, EF will try to set the foreign key to null.

I'm sure there's a reason for this behavior, but I don't understand it. If EF knows the foreign key can't be null, why not go ahead and delete the child entity?

If your child table has a non-identifying relationship (which is common), you have two options:

  1. Change the table's foreign key to a composite key, as well as the entity's configuration in EF.
  2. Only change the entity's EF configuration.

I like the second one. Here's how this looks.

Again, the non-Identifying Item relationship in the table

Order
-----
OrderId PK

Item
----
ItemId  PK
OrderId FK

Identifying Item relationship in Entify Framework

protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
    modelBuilder.Entity<Item>()
    .HasKey(a => new { a.ItemId, a.OrderId });
}

With this in place, using Order.Items.Remove(item) will result in Entity Framework deleting the item.

Foreign Keys That Are Manually Generated, And/Or a String, Int or GUID

Let's keep assuming a non-identifying relationship with non-nullable foreign keys. How does the EF configuration change if we're dealing with string or GUID primary keys? What if the keys aren't auto-generated?

I wrote a console application to explore this. It turns out each flavor needs a little tweak because of EF conventions.

Test Schema

Create a new database and run this script to create the tables.

--String Keys
create table Customers (
	CustomerId varchar(10) not null primary key,
)

create table Addresses (
	AddressId varchar(10) not null primary key,
	CustomerId varchar(10) not null foreign key references Customers(CustomerId),
)

--Manually-generated Int keys
create table Orders (
	OrderId int not null primary key,
)

create table Items (
	ItemId int not null primary key,
	OrderId int not null foreign key references Orders(OrderId),
)

--Identity Int keys
create table Projects (
	ProjectId int not null identity primary key,
)

create table Tasks (
	TaskId int not null identity primary key,
	ProjectId int not null foreign key references Projects(ProjectId),
)

--Manually-generated GUID keys
create table Stores (
	StoreId uniqueidentifier not null primary key,
)

create table Employees (
	EmployeeId uniqueidentifier not null primary key,
	StoreId uniqueidentifier not null foreign key references Stores(StoreId),
)

--"Identity" Default Sequential GUID keys
create table Gardens (
	GardenId uniqueidentifier default newsequentialid() not null primary key,
)

create table Flowers (
	FlowerId uniqueidentifier default newsequentialid() not null primary key,
	GardenId uniqueidentifier not null foreign key references Gardens(GardenId),
)

Classes

Here are the classes that map to the above tables.

//String Keys
public class Customer
{
    public string CustomerId { get; set; }
    public virtual ICollection<Address> Addresses { get; set; } = new HashSet<Address>();
}

public class Address
{
    public string AddressId { get; set; }
    public string CustomerId { get; set; }
    public Customer Customer { get; set; }
}

//Manually-generated Int keys
public class Order
{
    public int OrderId { get; set; }
    public virtual ICollection<Item> Items { get; set; } = new HashSet<Item>();
}

public class Item
{
    public int ItemId { get; set; }
    public int OrderId { get; set; }
    public Order Order { get; set; }
}

//Identity Int keys
public class Project
{
    public int ProjectId { get; set; }
    public virtual ICollection<Task> Tasks { get; set; } = new HashSet<Task>();
}

public class Task
{
    public int TaskId { get; set; }
    public int ProjectId { get; set; }
    public Project Project { get; set; }
}

//Manually-generated GUID keys
public class Store
{
    public Guid StoreId { get; set; }
    public ICollection<Employee> Employees { get; set; } = new HashSet<Employee>();
}

public class Employee
{
    public Guid EmployeeId { get; set; }
    public Guid StoreId { get; set; }
    public Store Store { get; set; }
}

//Identity GUID keys
public class Garden
{
    public Guid GardenId { get; set; }
    public ICollection<Flower> Flowers { get; set; } = new HashSet<Flower>();
}

public class Flower
{
    public Guid FlowerId { get; set; }
    public Guid GardenId { get; set; }
    public Garden Garden { get; set; }
}

Basic DbContext class

My DbContext is named EfTestDb. The basic setup lets us view SQL statements.

I've turned off lazy loading, because I'm finding that a typical enterprise practice. The configurations are the same if lazy loading is turned on.

public class EfTestDb : DbContext
{
    public DbSet<Customer> Customers { get; set; }
    public DbSet<Address> Addresses { get; set; }
    public DbSet<Order> Orders { get; set; }
    public DbSet<Item> Items { get; set; }
    public DbSet<Project> Projects { get; set; }
    public DbSet<Task> Tasks { get; set; }
    public DbSet<Store> Stores { get; set; }
    public DbSet<Employee> Employees { get; set; }
    public DbSet<Garden> Gardens { get; set; }
    public DbSet<Flower> Flowers { get; set; }

    public EfTestDb()
    {
        Database.SetInitializer<EfTestDb>(null);
        Configuration.LazyLoadingEnabled = false;
        Configuration.ProxyCreationEnabled = false;

        //Show SQL in Output Debug window.
        this.Database.Log = s =>
        {
            System.Diagnostics.Debug.Write(s);
            System.Diagnostics.Trace.Write(s);
            Console.WriteLine(s);
        };
    }
}

String Key (Manually Generated)

It's still pretty common to have primary keys that are "intelligent" strings such as "blue-washer-20050225" that are generated by the application.

Here's the EF configuration.

I'm relying on EF conventions for key column naming, otherwise I'd need more configurations for .HasKey, .HasColumnName, etc.

// # String Keys
modelBuilder.Entity<Customer>()
    .HasMany(a => a.Addresses);

modelBuilder.Entity<Address>()
    .HasKey(a => new { a.AddressId, a.CustomerId })
    .HasRequired(a => a.Customer);

This is the general pattern we'll see. The difference will be in how the keys and properties are defined.

Here's the testing code. I'm using DetachAllEntities to simulate saving a complex entity in one session, then deleting a child in another.

class Program
{
    static void Main(string[] args)
    {
        StringKey();
        ManualInt();
        IdentityInt();
        ManualGuid();
        IdentityGuid();

        Console.ReadLine();
    }

    static void DetachAllEntities(EfTestDb db)
    {
        foreach (var entry in db.ChangeTracker.Entries()) { entry.State = EntityState.Detached; }
    }

    static void StringKey()
    {
        using (var db = new EfTestDb())
        {
            //clean up
            db.Addresses.RemoveRange(db.Addresses);
            db.Customers.RemoveRange(db.Customers);
            db.SaveChanges();
            DetachAllEntities(db);

            //add complex entity
            var customer = new Customer()
            {
                CustomerId = "a",
                Addresses = new HashSet<Address>()
                {
                    new Address() { AddressId = "aa" },
                    new Address() { AddressId = "bb" }
                }
            };

            db.Customers.Add(customer);
            db.SaveChanges();
            DetachAllEntities(db);

            //delete child
            customer = db.Customers.Include(a => a.Addresses).First();
            var address = customer.Addresses.First();
            customer.Addresses.Remove(address);
            db.SaveChanges();
        }
    }
    //... remaining test methods
}

Manually-Generated Int keys

// # Manually-generated Int keys
modelBuilder.Entity<Order>()
    .HasMany(a => a.Items);
modelBuilder.Entity<Order>()
    // I don't know why this is required, except that 
    // maybe EF assumes an int key is going to be an identity column
    .Property(p => p.OrderId)
        .HasDatabaseGeneratedOption(System.ComponentModel.DataAnnotations.Schema.DatabaseGeneratedOption.None);

modelBuilder.Entity<Item>()
    .HasKey(a => new { a.ItemId, a.OrderId })
    .HasRequired(a => a.Order);

static void ManualInt()
{
    var db = new EfTestDb();

    //clean up
    db.Items.RemoveRange(db.Items);
    db.Orders.RemoveRange(db.Orders);
    db.SaveChanges();
    DetachAllEntities(db);

    //add complex entity
    var order = new Order()
    {
        OrderId = 1,
        Items = new List<Item>()
        {
            new Item() { ItemId = 11 },
            new Item() { ItemId = 22 }
        }
    };

    db.Orders.Add(order);
    db.SaveChanges();
    DetachAllEntities(db);

    //delete child
    order = db.Orders.Include(a => a.Items).First();
    var Item = order.Items.First();
    order.Items.Remove(Item);
    db.SaveChanges();
}

Identity Int keys

modelBuilder.Entity<Task>()
    .HasKey(a => new { a.TaskId, a.ProjectId });
modelBuilder.Entity<Task>()
    .Property(p => p.TaskId)
        // required because EF can't assume TaskId is an identity column since it's part of a composite key.
        .HasDatabaseGeneratedOption(System.ComponentModel.DataAnnotations.Schema.DatabaseGeneratedOption.Identity);
   static void IdentityInt()
    {
        var db = new EfTestDb();

        //clean up
        db.Tasks.RemoveRange(db.Tasks);
        db.Projects.RemoveRange(db.Projects);
        db.SaveChanges();
        DetachAllEntities(db);

        //add complex entity
        var Project = new Project()
        {
            Tasks = new List<Task>()
            {
                new Task(),
                new Task()
            }
        };

        db.Projects.Add(Project);
        db.SaveChanges();
        DetachAllEntities(db);

        //delete child
        Project = db.Projects.Include(a => a.Tasks).First();
        var Task = Project.Tasks.First();
        Project.Tasks.Remove(Task);
        db.SaveChanges();
    }

Manually-Generated GUID keys

modelBuilder.Entity<Store>()
    .HasMany(a => a.Employees);

modelBuilder.Entity<Employee>()
    .HasKey(a => new { a.EmployeeId, a.StoreId })
    .HasRequired(a => a.Store);
static void ManualGuid()
{
    var db = new EfTestDb();

    //clean up
    db.Employees.RemoveRange(db.Employees);
    db.Stores.RemoveRange(db.Stores);
    db.SaveChanges();
    DetachAllEntities(db);

    //add complex entity
    var Store = new Store()
    {
        StoreId = Guid.NewGuid(),
        Employees = new List<Employee>()
        {
            new Employee() { EmployeeId = Guid.NewGuid() },
            new Employee() { EmployeeId = Guid.NewGuid() }
        }
    };

    db.Stores.Add(Store);
    db.SaveChanges();
    DetachAllEntities(db);

    //delete child
    Store = db.Stores.Include(a => a.Employees).First();
    var Employee = Store.Employees.First();
    Store.Employees.Remove(Employee);
    db.SaveChanges();
}

"Identity" GUID (Default Sequential GUID keys)

modelBuilder.Entity<Garden>()
.Property(p => p.GardenId)
    .HasDatabaseGeneratedOption(System.ComponentModel.DataAnnotations.Schema.DatabaseGeneratedOption.Identity);
modelBuilder.Entity<Garden>()
    .HasMany(a => a.Flowers);

modelBuilder.Entity<Flower>()
.Property(p => p.FlowerId)
    .HasDatabaseGeneratedOption(System.ComponentModel.DataAnnotations.Schema.DatabaseGeneratedOption.Identity);
modelBuilder.Entity<Flower>()
    .HasKey(a => new { a.FlowerId, a.GardenId })
    .HasRequired(a => a.Garden);
static void IdentityGuid()
{
    var db = new EfTestDb();

    //clean up
    db.Flowers.RemoveRange(db.Flowers);
    db.Gardens.RemoveRange(db.Gardens);
    db.SaveChanges();
    DetachAllEntities(db);

    //add complex entity
    var Garden = new Garden()
    {
        Flowers = new List<Flower>()
        {
            new Flower(),
            new Flower()
        }
    };

    db.Gardens.Add(Garden);
    db.SaveChanges();
    DetachAllEntities(db);

    //delete child
    Garden = db.Gardens.Include(a => a.Flowers).First();
    var Flower = Garden.Flowers.First();
    Garden.Flowers.Remove(Flower);
    db.SaveChanges();
}

Lessons Learned

  • Non-identifying, but required, parent-child relationships are common.
  • The database doesn't have to be changed to allow Entity Framework to easily delete child records.
  • The most common key types can be modeled.

References

Other Background Information