One of the most popular posts on this blog is a very simple write-up on how to parse JSON in .NET Core. I mostly wrote it because I thought that there was definitely a “proper” way of doing things, and people were almost going out of their way to make life difficult for themselves when working with JSON.

I think working with XML is slightly different because (just IMO), there still isn’t a “gold standard” library for XML.

Unlike JSON which has the incredible JSON.NET library to handle everything and anything, the majority of cases when you work with XML you’ll use one of the inbuilt XML Parsers inside the .NET Core framework. These can be frustrating at times and incredibly brittle. Part of it is that they were created very early on in the creation of .NET, and because of that, always need to be backwards compatible so you lose out on things like Generics. The other part is that the actual XML spec that involves things like namespaces and DTDs, while at first look simple, can be incredibly harsh. By harsh I mean that things will just plain not work if you are missing just one piece of the puzzle, and it can take hours to work out what’s wrong.

Anyway, let’s jump right in and check out our options for working with XML in .NET.

Our Example XML File

I’m going to be using a very simple XML file that has an element, an attribute property and a list. I’ll use these as we check out the options so we are always comparing trying to read the same file.

<?xml version="1.0" encoding="utf-8" ?>
<MyDocument xmlns="http://www.dotnetcoretutorials.com/namespace">
  <MyProperty>Abc</MyProperty>
  <MyAttributeProperty value="123" />
  <MyList>
    <MyListItem>1</MyListItem>
    <MyListItem>2</MyListItem>
    <MyListItem>3</MyListItem>
  </MyList>
</MyDocument>

Using XMLReader

So the first option we have is using the class “XMLReader”. It’s a forward only XML Parser (By that I mean that you read the file line by line almost). I’ll warn you now, it’s very very primitive. For example our code might look a bit like so :

XmlReaderSettings settings = new XmlReaderSettings();
settings.IgnoreWhitespace = true;

using (var fileStream = File.OpenText("test.xml"))
using(XmlReader reader = XmlReader.Create(fileStream, settings))
{
    while(reader.Read())
    {
        switch(reader.NodeType)
        {
            case XmlNodeType.Element:
                Console.WriteLine($"Start Element: {reader.Name}. Has Attributes? : {reader.HasAttributes}");
                break;
            case XmlNodeType.Text:
                Console.WriteLine($"Inner Text: {reader.Value}");
                break;
            case XmlNodeType.EndElement:
                Console.WriteLine($"End Element: {reader.Name}");
                break;
            default:
                Console.WriteLine($"Unknown: {reader.NodeType}");
                break;
        }
    }
}

With the output looking like :

Unknown: XmlDeclaration
Start Element: MyDocument. Has Attributes? : True
Start Element: MyProperty. Has Attributes? : False
Inner Text: Abc
End Element: MyProperty
Start Element: MyAttributePropety. Has Attributes? : True
Start Element: MyList. Has Attributes? : False
Start Element: MyListItem. Has Attributes? : False
Inner Text: 1
End Element: MyListItem
Start Element: MyListItem. Has Attributes? : False
Inner Text: 2
End Element: MyListItem
Start Element: MyListItem. Has Attributes? : False
Inner Text: 3
End Element: MyListItem
End Element: MyList
End Element: MyDocument

It sort of reminds me of using ADO.NET and reading data row by row and trying to store it in an object. The general idea is because you are only parsing line by line, it’s less memory intensive. But you’re also having to handle each line individually with any number of permutations of elements/attributes/lists etc. I think the only reason to use this method would be if you have extremely large XML files (100+MB), or you are looking for something very very specific. e.g. you only want to read a single element from the file, and you don’t want to load the entire thing while looking for that one element.

Another thing I will point out is that XML Namespaces and the difficulty around those wasn’t there with XMLReader. It just sort of powered through and there wasn’t any issue around prefixes, namespaces, DTDs etc.

But again in general, I wouldn’t use XMLReader in the majority of cases.

Using XPathDocument/XPathNavigator

So another way of getting individual XML Nodes, but being able to “search” a document is using the XPathNavigator object.

First, the code :

using (var fileStream = File.Open("test.xml", FileMode.Open))
{
    //Load the file and create a navigator object. 
    XPathDocument xPath = new XPathDocument(fileStream);
    var navigator = xPath.CreateNavigator();

    //Compile the query with a namespace prefix. 
    XPathExpression query = navigator.Compile("ns:MyDocument/ns:MyProperty");

    //Do some BS to get the default namespace to actually be called ns. 
    var nameSpace = new XmlNamespaceManager(navigator.NameTable);
    nameSpace.AddNamespace("ns", "http://www.dotnetcoretutorials.com/namespace");
    query.SetContext(nameSpace);

    Console.WriteLine("My Property Value : " + navigator.SelectSingleNode(query).Value);
}

Now honestly… This is bad and I made it bad for a reason. Namespaces here are really painful. In my particular case because I have a default namespace, this was the only way I could find out there that would get the XPath working. Without the namespace, things would actually be a cinch. So with that said I’m going to admit something here… I have totally used string replace functions to remove namespaces before… Now I know someone will jump in the comments and say “but the XML spec says blah blah blah”. I honestly think every headache I’ve ever had with working with XML has been because of namespaces.

So let me put a caveat on my recommendation here. If the document you are working with does not make use of namespaces (Or you are willing to remove them), and you need use an XPath expression to get a single node, then using the XMLNavigator actually isn’t a bad option. But that’s a big if.

Using XMLDocument

XMLDocument can be thought of like an upgraded version of the XPathNavigator. It has a few easier methods to load documents, and allows you to modify XMLDocuments in memory too!

XmlDocument document = new XmlDocument();
document.Load("test.xml");

XmlNamespaceManager m = new XmlNamespaceManager(document.NameTable);
m.AddNamespace("ns", "http://www.dotnetcoretutorials.com/namespace");
Console.WriteLine(document.SelectSingleNode("ns:MyDocument/ns:MyProperty", m).InnerText);

Overall you still have to deal with some namespace funny business (e.g. Default Namespaces are not handled great), and you still have to get each element one by one as you need it, but I do think this is the best option if you are looking to load out only a small subset of the XML doc. The fact you can modify the XML and save it back to file is also a pretty good one.

Using XMLSerializer

Now we are cooking with gas, XMLSerializer in my opinion is the very best way to parse XML in .NET Core. If you’ve used JSONDocument from JSON.NET before, then this is very close to being the same sort of setup.

First we simply create a class that models our actual XML file. We use a bunch of attribute to specify how to read the doc, which namespace we are using, even what type of element we are trying to deserialize (e.g. An attribute, element or array).

[XmlRoot("MyDocument", Namespace = "http://www.dotnetcoretutorials.com/namespace")]
public class MyDocument
{
    public string MyProperty { get; set; }

    public MyAttributeProperty MyAttributeProperty { get; set; }

    [XmlArray]
    [XmlArrayItem(ElementName = "MyListItem")]
    public List MyList { get; set; }
}

public class MyAttributeProperty
{
    [XmlAttribute("value")]
    public int Value { get; set; }
}

Really really simple. And then the code to actually read our XML and turn it into this class :

using (var fileStream = File.Open("test.xml", FileMode.Open))
{
    XmlSerializer serializer = new XmlSerializer(typeof(MyDocument));
    var myDocument = (MyDocument)serializer.Deserialize(fileStream);

    Console.WriteLine($"My Property : {myDocument.MyProperty}");
    Console.WriteLine($"My Attribute : {myDocument.MyAttributeProperty.Value}");

    foreach(var item in myDocument.MyList)
    {
        Console.WriteLine(item);
    }
}

No messing about trying to get namespaces right, no trying to work out the correct XPath, it just works. I think once you start using XMLSerializer, you will wonder why you ever bothered trying to manually read out XML documents again.

Now there is a big caveat. If you don’t really care about the bulk of the document and you are just trying to get a really deep element, it can be painful creating these huge models and classes just go get a single element.

Overall, in 99.9% of cases, try and use XMLSerializer to parse XML. It’s less brittle than other options and follows a very similar “pattern” to that of JSON serialization meaning anyone who has worked with one, can work with the other.

I skipped out on University/College in my earlier years and instead opted to fully self teach myself programming. While that means I can chew someone’s ear off about the latest feature in C# 8, it also means that I missed out on plenty of “Data Structure/Algorithm” style programming problems. Let’s be honest, unless you are going for a job interview it’s rare you use these anyway – especially in programming business systems. But every now and again I come across an algorithm that I think “How have I not heard of this before?”. One such algo is the “Knapsack Problem”, also sometimes known as the “Rucksack Problem”.

One of the biggest issues I found with looking for an algorithm in C# to do this, is very rarely did they ever explain how the code worked, or even a thorough explanation on how to use it. So this is that explanation.

What Is The Knapsack Problem?

The Knapsack Problem is where you have a “bag” that can hold a limited number of items, given that you have a set of items to choose from each with individual “values”, how can you maximize filling your bag with only the most valuable items.

Let’s take a real world example. A robber has broken into a jewellery store and wants to steal precious jewellery. His backpack can only hold 50KG of weight (He’s superman). When he walks around the store thinking about what to steal, he has to do a sort of “cost/benefit” sum in his head to maximize his total take. As an example, with a 50KG bag, is it better to steal a 50KG item worth $100, or steal five 10kg items worth $50 each? Obviously the latter because even though that one item is more valuable, it takes up all the space in the bag, whereas stealing multiple smaller items actually maximizes the value the bag can hold. This is the knapsack/rucksack problem.

Code Explanation

Before I just give you the code, I want to explain a little bit first on how this actually works. Even though the code below has a tonne of comments, I still want to go that extra mile because it can be hard to wrap your head around at first.

The code works a bit like this :

• Create a loop to go through each jewel one by one.
• Given a smaller bag, and increasing the bag size each loop, check if the jewel can fit inside the bag (Even if we had to empty out something else)
• If the Jewel can fit, then work out what’s already in the “bag” by checking what the last value was when we looped at this bag size. Or another way of putting it, on the last jewel we looped on, when we were at this weight, what was the value? And with the current jewel we have, it we combine it with other jewels to make up the other space in the bag, who has the higher value? If it’s the current Jewel + extra then set a matrix value to that, otherwise set it to the last jewel we fit at this weight.
• If the jewel can’t fit into our smaller bag (e.g.  The jewel weighs 30KG but our bag can only carry 20KG, then take whatever the last jewel round was that fit into this 20KG bag).
• As we carry values forward, then at the very end, the last indexes in our matrix will be the maximum value.

… I’m going to stop here and say that this is probably a little confusing. So let’s take an example of where we might be in the current loop cycle and how this might look by building a quick flow diagram. (Click to open to a larger image).

Knapsack Code

Now here’s the code. Note that this is not crazy optimized with one letter variables names for a leetcode competition, but it’s written in a way that hopefully is easier to understand.

public class Jewel
{
	public int Weight { get; set; }
	public int Value { get; set; }
}

public static int KnapSack(int bagCapacity, List jewels)
{
	var itemCount = jewels.Count;

	int[,] matrix = new int[itemCount + 1, bagCapacity + 1];

	//Go through each item. 
	for (int i = 0; i <= itemCount; i++)
	{
		//This loop basically starts at 0, and slowly gets bigger. 
		//Think of it like working out the best way to fit into smaller bags and then keep building on that. 
		for (int w = 0; w <= bagCapacity; w++)
		{
			//If we are on the first loop, then set our starting matrix value to 0. 
			if (i == 0 || w == 0)
			{
				matrix[i, w] = 0;
				continue;
			}

			//Because indexes start at 0, 
			//it's easier to read if we do this here so we don't think that we are reading the "previous" element etc. 
			var currentJewelIndex = i - 1;
			var currentJewel = jewels[currentJewelIndex];

			if (i == 0 || w == 0)
				matrix[i, w] = 0;
			//Is the weight of the current jewel less than W 
			//(e.g. We could find a place to put it in the bag if we had to, even if we emptied something else?)
			if (currentJewel.Weight <= w)
			{
				//If I took this jewel right now, and combined it with other gems
				//Would that be bigger than what you currently think is the best effort now? 
				//In other words, if W is 50, and I weigh 30. If I joined up with another jewel that was 20 (Or multiple that weigh 20, or none)
				//Would I be better off with that combination than what you have right now?
				//If not, then just set the value to be whatever happened with the last item 
				//(may have fit, may have done the same thing and not fit and got the previous etc). 
				matrix[i, w] = Math.Max(currentJewel.Value + matrix[i - 1, w - currentJewel.Weight]
										, matrix[i - 1, w]);
			}
			//This jewel can't fit, so bring forward what the last value was because that's still the "best" fit we have. 
			else
				matrix[i, w] = matrix[i - 1, w];
		}
	}

	//Because we carry everything forward, the very last item on both indexes is our max val
	return matrix[itemCount, bagCapacity];
}

static void Main(string[] args)
{
	var items = new List
		{
			new Jewel {Value = 120, Weight = 10},
			new Jewel {Value = 100, Weight = 20},
			new Jewel {Value = 500, Weight = 30},

		};

	Console.WriteLine(KnapSack(50, items));
}

I’ve included a sample little console debug (So you can paste this into a new console app), to illustrate how it works. Again, I’ve commented and used a “Jewel” class to illustrate the example a little more so that hopefully it’s easier to understand than some code golf example. Even if you don’t use C# as your main language, hopefully it’s easy to follow!

I had a friend who was taking a look through the classic “Gang Of Four” Design Patterns book for the first time. He reached out to ask me which of the design patterns I’ve actually used in business applications, and actually thought “I’m using this pattern right now”. Singleton, Factory Pattern, Mediator – I’ve used all of these and I’ve even written about them before. But one that I haven’t talked about before is the Chain Of Responsibility Pattern.

What Is “Chain Of Responsibility”

Chain Of Responsibility (Or sometimes I’ve called it Chain Of Command) pattern is a design pattern that allows “processing” of an object in hierarchical fashion. The classic Wikipedia definition is

In object-oriented design, the chain-of-responsibility pattern is a design pattern consisting of a source of command objects and a series of processing objects. Each processing object contains logic that defines the types of command objects that it can handle; the rest are passed to the next processing object in the chain. A mechanism also exists for adding new processing objects to the end of this chain. Thus, the chain of responsibility is an object oriented version of the if … else if … else if ……. else … endif idiom, with the benefit that the condition–action blocks can be dynamically rearranged and reconfigured at runtime.

That probably doesn’t make much sense but let’s look at a real world example that we can then turn into code.

Let’s say I own a bank. Inside this bank I have 3 levels of employees. A Bank Teller, Supervisor, and a Bank Manager. If someone comes in to withdraw money, the Teller can allow any withdrawal of less than $10,000, no questions asked. If the amount is for more than $10,000, then it passes the request onto the supervisor. The supervisor can handle requests up to $100,000, but only if the account has ID on record. If the ID is not on record, then the request must be rejected no matter what. If the requested amount is for more than $100,000 it goes to the bank manager. The bank manager can approve any amount for withdrawal even if the ID is not on record because if they are withdrawing that amount, they are a VIP and we don’t care about ID and money laundering regulations.

This is the hierarchical “Chain” that we talked about earlier where each person tries to process the request, and can then pass it onto the next. If we take this approach and map it to code (In an elegant way), this is what we call the Chain Of Responsibility pattern. But before we go any further, let’s look at a bad way to solve this problem.

A Bad Approach

Let’s just solve this entire problem using If/Else statements.

class BankAccount
{
    bool idOnRecord { get; set; }

    void WithdrawMoney(decimal amount)
    {
        // Handled by the teller. 
        if(amount < 10000)
        {
            Console.WriteLine("Amount withdrawn by teller");
        } 
        // Handled by supervisor
        else if (amount < 100000)
        {
            if(!idOnRecord)
            {
                throw new Exception("Account holder does not have ID on record.");
            }

            Console.WriteLine("Amount withdrawn by Supervisor");
        }
        else
        {
            Console.WriteLine("Amount withdrawn by Bank Manager");
        }
    }
}

So there is a few issues with our code.

• Adding additional levels of employees in here is really hard to manage with the mess of If/Else statements.
• The special logic of checking ID at the supervisor level is somewhat hard to unit test because it has to pass a few other checks first.
• While the only defining logic is for the amount withdrawn at the moment, we could add additional checks in the future (e.g. VIP customers are marked as such and are always handled by the supervisor). This logic is going to be hard to manage and could easily get out of control.

Coding Chain Of Responsibility

Let’s rewrite the code a little. Instead let’s create “employee” objects that can handle the logic of whether they can process the request themselves or not. Ontop of that, let’s give them a line manager so that they know they can pass the request up if needed.

interface IBankEmployee
{
    IBankEmployee LineManager { get; }
    void HandleWithdrawRequest(BankAccount account, decimal amount);
}

class Teller : IBankEmployee
{
    public IBankEmployee LineManager { get; set; }

    public void HandleWithdrawRequest(BankAccount account, decimal amount)
    {
        if(amount > 10000)
        {
            LineManager.HandleWithdrawRequest(account, amount);
            return;
        }

        Console.WriteLine("Amount withdrawn by Teller");
    }
}

class Supervisor : IBankEmployee
{
    public IBankEmployee LineManager { get; set; }

    public void HandleWithdrawRequest(BankAccount account, decimal amount)
    {
        if (amount > 100000)
        {
            LineManager.HandleWithdrawRequest(account, amount);
            return;
        }

        if(!account.idOnRecord)
        {
            throw new Exception("Account holder does not have ID on record.");
        }

        Console.WriteLine("Amount withdrawn by Supervisor");
    }
}

class BankManager : IBankEmployee
{
    public IBankEmployee LineManager { get; set; }

    public void HandleWithdrawRequest(BankAccount account, decimal amount)
    {
        Console.WriteLine("Amount withdrawn by Bank Manager");
    }
}

We can then create the “chain” by creating the employees required along with their managers. Almost like creating an Org Chart.

var bankManager = new BankManager();
var bankSupervisor = new Supervisor { LineManager = bankManager };
var frontLineStaff = new Teller { LineManager = bankSupervisor };

We can then completely transform the BankAccount class Withdraw method to instead be handled by our front line staff member (The Teller).

class BankAccount
{
    public bool idOnRecord { get; set; }

    public void WithdrawMoney(IBankEmployee frontLineStaff, decimal amount)
    {
            frontLineStaff.HandleWithdrawRequest(this, amount);
    }
}

Now, when we make a withdrawl request, the Teller always handles it first, if it can’t, it then passes it to it’s line manager *whoever* that may be. So the beauty of this pattern is

• Subsequent items in the “chain” don’t need to know why things got passed to it. A supervisor doesn’t need to know what the requirements for on why a Teller passed it up the chain.
• A Teller doesn’t need to know the entire chain after it. Just that it passed the request to the supervisor and it will be handled there (Or further if need be).
• The entire org chart can be changed by introducing new employee types. For example if I created a “Teller Manager” that could handle requests between 10k -> 50k, and then pass it to the Supervisor. The Teller object would stay the same, The Supervisor object would stay the same, and I would just change the LineManager of the Teller to be the “Teller Manager” instead.
• Any Unit Tests we write can focus on a single employee at once. For example when testing a Supervisor, we don’t also need to test the Teller’s logic on when it gets passed to it.

Extending Our Example

While I think the above example is a great way to illustrate the pattern, often you’ll find people using a method called “SetNext”. In general I think this is pretty uncommon in C# because we have property getters and setters. Using a “SetVariableName” method is typically from C++ (And for me – Pascal) days where that was the preferred way of encapsulating variables.

But ontop of that, other examples also typically use an Abstract Class to try and tighten how requests are passed along. The problem with our example above is that there is a lot of duplicate code of passing the request onto the next handler. Let’s tidy that up a little bit.

There is a lot of code so bare with me. The first thing we want to do is create an AbstractClass that allows us to handle the withdrawal request in a standardized way. It should check the condition, if it passes, do the withdraw, if not, it needs to pass it onto it’s line manager. That looks like so :

interface IBankEmployee
{
    IBankEmployee LineManager { get; }
    void HandleWithdrawRequest(BankAccount account, decimal amount);
}

abstract class BankEmployee : IBankEmployee
{
    public IBankEmployee LineManager { get; private set; }

    public void SetLineManager(IBankEmployee lineManager)
    {
        this.LineManager = lineManager;
    }

    public void HandleWithdrawRequest(BankAccount account, decimal amount)
    {
        if (CanHandleRequest(account, amount))
        {
            Withdraw(account, amount);
        } else
        {
            LineManager.HandleWithdrawRequest(account, amount);
        }
    }

    abstract protected bool CanHandleRequest(BankAccount account, decimal amount);

    abstract protected void Withdraw(BankAccount account, decimal amount);
}

Next we need to modify our employee classes to inherit from this BankEmployee class.

class Teller : BankEmployee, IBankEmployee
{
    protected override bool CanHandleRequest(BankAccount account, decimal amount)
    {
        if (amount > 10000)
        {
            return false;
        }
        return true;
    }

    protected override void Withdraw(BankAccount account, decimal amount)
    {
        Console.WriteLine("Amount withdrawn by Teller");
    }
}

class Supervisor : BankEmployee, IBankEmployee
{
    protected override bool CanHandleRequest(BankAccount account, decimal amount)
    {
        if (amount > 100000)
        {
            return false;
        }
        return true;
    }

    protected override void Withdraw(BankAccount account, decimal amount)
    {
        if (!account.idOnRecord)
        {
            throw new Exception("Account holder does not have ID on record.");
        }

        Console.WriteLine("Amount withdrawn by Supervisor");
    }
}

class BankManager : BankEmployee, IBankEmployee
{
    protected override bool CanHandleRequest(BankAccount account, decimal amount)
    {
        return true;
    }

    protected override void Withdraw(BankAccount account, decimal amount)
    {
        Console.WriteLine("Amount withdrawn by Bank Manager");
    }
}

So notice that in all cases, the public method of “HandleWithdrawRequest” from the abstract class is called, it then calls the subclass “CanHandleRequest” which can contain our logic on if this employee is good to go or not. If it is, then call it’s local “Withdraw” request, otherwise try the next employee.

We just need to change how we create the chain of employees like so :

var bankManager = new BankManager();

var bankSupervisor = new Supervisor();
bankSupervisor.SetLineManager(bankManager);

var frontLineStaff = new Teller();
frontLineStaff.SetLineManager(bankSupervisor);

Again, I prefer not to use the “SetX” methods, but it’s what a lot of examples use so I thought I would include it.

Other examples also put the logic of whether an employee can handle the request or not inside the actual abstract class. I personally prefer not to do this as it means all our handlers have to have very similar logic. So for example at the moment all are checking the amount to be withdrawn, but what if we had a particular handler that was looking for something in particular (Like a VIP flag?), adding that logic into the abstract class for some handlers but not others would just take us back to If/Else hell.

When To Use The “Chain Of Responsibility” Design Pattern?

The best use cases of this pattern are where you have a very logical “chain” of handlers that should be run in order every time. I would note that forking of the chain is a variation on this pattern, but quickly becomes extremely complex to handle. For that reason, I typically end up using this pattern when I am modelling real world “chain of command” scenarios. It’s the entire reason I use a bank as an example, because it’s a real world “Chain Of Responsibility” that can be modelled in code.

As countries head into lockdown/quarantine/#StayHomeSaveLives, we find ourselves with a fair bit of free time on our hands. While I’m sure Netflix, Disney Plus,  and Amazon Prime will be getting a thorough workout during this time, it’s also a great opportunity to upskill and maybe gain a couple of those certifications you’ve been meaning to get for the past few years now. Even if your aim isn’t to actually sit an exam, there’s no harm in using all this time to actually upskill and hit the ground running when this is all over.

So here’s a select few courses that I’ve personally done and can recommend. Some are programming, some are devops, some are Scrum (ugh, I know but.. hear me out). A few have some free study options but even if there is a cost, typically it’s only going to be ~$10 or so. And even if you don’t use my recommended course, often there are free alternatives out there. So let’s jump right in!

Azure Exam AZ-204 – Developing Solutions For Microsoft Azure

So let’s start off with one that I’m sure many readers will think about doing in their career. The Microsoft exam AZ-204 is made for developers to expand their knowledge on everything Azure. AZ-204 used to be called AZ-203, and before that was 70-532, but they are all roughly the same thing. The only difference is that as Azure keep adding new services and features, the exam has to keep getting updated. Because the exam outline is released once (When the exam is released), they can’t just keep adding into the same exam and have to update the number. Regardless, AZ-204, AZ-203 and 70-532 are all on developers getting to grips with Azure services.

Who Is This For?

If you are a developer and use Azure for work, sit this exam. It’s that simple. I would even go as far to say that even if you are a tester/QA, or someone that is an “almost” developer (Like DevOps, report writer, SQL developer etc), you should sit this exam. If you aren’t someone who likes exams, following the study materials is still an amazing way to learn the ins and outs of a tonne of Azure services. I can’t even begin to describe the amount of times I’ve been asked a question at work and thought “huh… I have the exact answer just from flicking through that Azure book for the past few weeks”.

Exam Cost

The exam itself is around $165USD and as of right now, you can sit the exam from your home. You do not need to go into a testing center to complete. So you can actually complete all studying and sit the exam during the lockdown!

Check out the exam here : https://docs.microsoft.com/en-gb/learn/certifications/exams/az-204

Study Materials

There are a multitude of ways to study for this exam, so I’ll give a quick rundown of them all.

If you like studying from books, there are Microsoft Exam Ref books available on Amazon. Unfortunately a book for AZ-204 hasn’t been released so you would have to use the AZ-203 reference book (Kindle/Paperback on Amazon here). I personally bought the Exam Ref 70-532 book (Kindle/Paperback on Amazon here), and thought it was a bit of a mixed bag. There was typos and references to non existent diagrams all over the place, but it was still a pretty solid book. The bonus of the Exam Ref book is that it gives you a bit of an intro to a topic in a structured way, and then I always just went and read more on the official Azure documentation.

If you prefer online video courses, then for pretty much all Azure exams, people use the series from Scott Duffy which are available on Udemy here. I’ve personally used Scott Duffy’s stuff for both the developer and architecture exams and found them pretty much spot on. For the cost ($~14), it’s kinda a no brainer. The other thing I’ll mention is that I bought the course when the exam was still 70-532 and I still have access to the AZ-203. Scott gives all existing students access to the updated exams. So even if you buy the AZ-203 course, when that gets rolled into AZ-204, you will get access to that as well. I mean that’s not a guarantee but so far, I purchased the course in February of 2017 and I still have access right now. It’s crazy good value.

Finally, Microsoft have free learning resources available that are typically a mix of articles/videos available at https://docs.microsoft.com/en-gb/learn/browse/ and https://channel9.msdn.com/Series/Microsoft-Azure-Tutorials. These are great free resources but I’ve never really found them to be that aligned to the exam. In some ways that’s a good thing since you aren’t learning specifically what’s in the exam and instead just working your way through Azure resources. But if you intend on sitting the exam, I recommend trying to stick with an Exam Ref course/book so that you know exactly what it is you should be learning.

Professional Scrum Master (PSM) – Scrum.org

Professional Scrum Master is the certification available from scrum.org. There is another certification in the scrum world called “Certified Scrum Master” that you will probably see scrum masters tack onto the end of their email signatures like it’s some sort of PHD, but they are testing the exam same thing. PSM is just an online exam for when you think you know the material. CSM is a course you have to do in person and pay a few grand for the badge of honor. In the grand scheme of things, knowing scrum is highly advantageous to the programming world right now, irrespective of the letters you want to put in your LinkedIn name.

Who Is This For?

I actually recommend sitting the PSM exam for anyone who works in a team (or even near a team) that works in any agile methodology. You don’t have to be angling for a scrum masters job to sit this exam, it really is for anyone looking to gain broader knowledge on exactly how scrum works. I found it extremely helpful in understanding the “why” of Scrum. “Why” do we even bother with a sprint retrospective? If someone is away, why should we even bother doing a standup that day?

Another reason I found this exam helpful was that you only learn exactly what the scrum guide actually says. The scrum guide itself is 15 odd pages of info and that’s it. It seems like every workplace has their own “additions” to Scrum or their own ScrumButs, so it’s good to separate out workplace frameworks versus what the scrum guide actually says.

Exam Cost

The exam is $150 and is purely an online exam. You can purchase it and sit it whenever you want from the comfort of your own home.

Study Materials

The incredible thing about learning scrum is you can download the scrum guide here, read it 3 times over, and sit the exam and pass. I personally think that if you have *never* used Scrum before it’s actually easier to sit the exam and pass because you only know what the guide told you. I’ve seen colleagues actually fail the exam because they think “here’s how we solve this problem at my work” rather than “the scrum guide says”.

I also used an online video course on Udemy called “Scrum Certification Prep“. While there is hours of lectures, I kinda felt like they were all over the place. When I did it, there were two different instructors with one being really hard to hear. It also seemed like a lot of the content was repetitive which… Is kinda unique to scrum in a way. Again, the scrum guide is so short and concise but people have made careers on running workshops for Scrum, and the only way to do that is to repeat the same things over and over.

AWS Cerified Solutions Architect Associate

While I recommended the developer exam for Azure, I actually recommend the architecture exam when it comes to AWS. I found the AWS Architecture Associate exam to be more of an AWS 101 rather than getting down to the nitty gritty. Personally, most of my work ends up on Azure just because of the synergy between the .NET ecosystem and things like Azure Web Apps. But I always want to keep up to date with what’s happening on AWS so that if we run into a road block in Azure, there’s always the “let’s check out what AWS offers”. Even if you never work with AWS, I think it’s helpful to know what else is out there at a high level – and the AWS Architecture Associate exam gives that.

Who Is This For?

If you’re working on AWS day to day I think that this exam is a no brainer from developers, to testers, to architects. If you are working mostly on Azure, then I think this exam/study is still beneficial but mostly if you are in a tech lead/architect sort of role where you always want to have a comparison of “this is what the other cloud offers”.

Exam Cost

All associate exams at AWS are $150USD each. Again AWS offers online exams so you can sit the exam from the comfort of your own home even during lockdown.

Study Materials

I’ve spoken to numerous people who have sat AWS exams and they all say one thing. “We used A Cloud Guru”. To give you an idea of how popular A Cloud Guru’s study material is, there are currently 520k students enrolled on the Udemy course here. That’s insane. Again, I purchased this course in November of 2016, and I still have access to the updates materials which is just crazy value for money.

A Cloud Guru also offer practice tests with over 200 questions for around $20 as well which is crazy value for money. You can grab the practice exams here. I highly recommend if you are looking to sit the exam, to do the practice questions as they may hit on some topics that you don’t actually know that well, giving you the opportunity to go and study that area a bit more.

I’ve debated about posting an article on this for a long long time. Mostly because I think if you code any C# MVC/API project correctly, you should almost never run into this issue, but I’ve seen people run into the problem over and over again. What issue? Well it’s when you end up with two very interesting exceptions. I say two because depending on whether you are using System.Text.Json or Newtonsoft.JSON in your API, you will get two different error messages.

For System.Text.Json (Default for .NET Core 3+), you would see something like :

JsonException: A possible object cycle was detected which is not supported. This can either be due to a cycle or if the object depth is larger than the maximum allowed depth of 32.

And for Newtonsoft.Json (Or JSON.NET as it’s sometimes called, default for .NET Core 2.2 and lower) :

JsonSerializationException: Self referencing loop detected with type

They mean essentially the same thing, that you have two models that reference each other and will cause an infinite loop of serializing doom.

Why Does This Happen?

Before we get into how to resolve the problem, let’s have a good dig into why this happens.

Let’s assume I have an API that contains the following models :

public class StaffMember
{
    public string FirstName { get; set; }
    public Department Department { get; set; }
}

public class Department
{
    public List StaffMembers { get; set; }
}

Already we can see that there is a small problem. The class StaffMember references Department, and Department references StaffMember. In normal C# code this isn’t such an issue because it’s simply pointers. But when we are working inside an API, when we output this model it has to traverse the full model to output our JSON model.

So if for example we had an API endpoint that looked like this :

[HttpGet]
public ActionResult Get()
{
    var staff = new StaffMember { FirstName = "John Smith" };
    var department = new Department();
    staff.Department = department;
    department.StaffMembers = new List { staff };

    return Ok(staff);
}

We are gonna blow up.

It takes our StaffMember to try and serialize, which points to the Department. It goes to the Department and tries to serialize it, and it finds a StaffMember. It follows that StaffMember and… goto:Step 1.

But you’re probably sitting there thinking “When have I ever created models that reference each other in such a way?!”. And you’d probably be pretty right. It’s exceedingly rare that you create these sorts of two way relationships… Except of course… when using Entity Framework relationships.

If these two classes were part of an EntityFramework model, it’s highly likely that it would look like so :

public class StaffMember
{
    public string FirstName { get; set; }
    public virtual Department Department { get; set; }
}

public class Department
{
    public virtual ICollection StaffMembers { get; set; }
}

In EntityFramework (Or many other ORMs), we create two way relationships because we want to be able to traverse models both ways, and often this can create reference loops.

The Real Solution

So putting “real” in the subtitle may trigger some because this isn’t so much a “here’s the line of code to fix this problem” solution in so much as it’s a “don’t do it in the first place”.

The actual problem with the above example is that we are returning a model (our datamodel) that isn’t fit to be serialized in the first place. In general terms, you *should not* be returning your data model direct from an API. In almost all cases, you should be returning a ViewModel from an API. And then when returning a view model, you wouldn’t make the same self referencing issue.

For example (Sorry for the long code)

[HttpGet]
public ActionResult Get()
{
    var staffMember = new StaffMember { Department = new Department() }; //(Really this should actually be calling a repository etc). 

    var viewModel = new StaffMemberViewModel
    {
        FirstName = staffMember.FirstName,
        Department = new StaffMemberViewModelDepartment
        {
            DepartmentName = staffMember.Department.DepartmentName
        }
    };

    return Ok(viewModel);
}

public class StaffMemberViewModel
{
    public string FirstName { get; set; }
    public StaffMemberViewModelDepartment Department { get; set; }
}

public class StaffMemberViewModelDepartment
{
    public string DepartmentName { get; set; }
}

public class StaffMember
{
    public string FirstName { get; set; }
    public virtual Department Department { get; set; }
}

public class Department
{
    public string DepartmentName { get; set; }
    public virtual ICollection StaffMembers { get; set; }
}

Here we can see we map the StaffMember data model into our fit for purpose ViewModel. It may seem overkill to create new models to get around this issue, but the reality is that this is best practice in any case. Not using ViewModels and instead returning your exact DataModel is actually going to cause a whole heap of other issues, so even if you solve the reference loop another way, you are still gonna have issues.

Global API Configuration Settings

So I’ve just ranted about how you really shouldn’t run into this issue if you use proper view models, but let’s say that you have to use a model that contains a reference loop. You have no other option, what can you do?

NewtonSoft.Json (JSON.NET)

Let’s start with if you are using Newtonsoft.Json first (If you are using .NET Core 3+, there is a guide on adding Newtonsoft back as the default JSON serializer here : https://dotnetcoretutorials.com/2019/12/19/using-newtonsoft-json-in-net-core-3-projects/).

You can then edit your startup.cs where you add in Newtonsoft to configure the ReferenceLoopHandling :

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers().AddNewtonsoftJson(x => x.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore);
}

Now when Newtonsoft.Json runs into a loop, it simply stops serializing that tree. To me it’s still not pretty as your output essentially ends up like :

{"firstName":"John Smith","department":{"departmentName":null,"staffMembers":[]}}

Anyone reading this would at first think that the Department has no Staff Members, but in reality it just stopped serializing at that point because it detected it was about to loop.

Another thing to note is if you set the serializer to instead Serialize like so :

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers().AddNewtonsoftJson(x => x.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Serialize);
}

Your website will just straight up crash. Locally, you will get an error like

The program '[23496] iisexpress.exe' has exited with code -1073741819 (0xc0000005) 'Access violation'.

But it’s essentially crashing because it’s looping forever (Like you’ve told it to!)

System.Text.Json

If you are using System.Text.Json, you are basically out of luck when it comes to reference loop support. There are numerous tickets created around this issue, but the actual feature is being tracked here : https://github.com/dotnet/runtime/issues/30820

Now the gist of it is, that the feature has been added to the serializer, but it’s not in the general release yet (Only preview release), and on top of that, even if you add the preview release to your project it actually doesn’t work as a global setting (See last comments on the issue). It only works if you manually create a serializer (For example serializing a model for an HttpClient call). So in general, if you are running into these issues and you don’t want to edit your ViewModel, for now it looks like you will have to use Newtonsoft.Json. I will update this post when System.Text.Json gets it’s act together!

Ignoring Properties

This one is weird because if you can access the model to make these changes, then just create a damn viewmodel! But in anycase, there is another way to avoid reference loops and that is to tell the serializer to not serialize a property at all. In *both* Newtonsoft.Json and System.Text.Json there is an attribute called JsonIgnore :

public class StaffMember
{
    public string FirstName { get; set; }
    public virtual Department Department { get; set; }
}

public class Department
{
    public string DepartmentName { get; set; }
    [JsonIgnore]
    public virtual ICollection StaffMembers { get; set; }
}

This means that the StaffMembers property on the Department object will simply not be output. It’s in my mind a slightly better option than above because you don’t see the property of StaffMembers with an empty array, instead you don’t see it at all!

I recently came across a peculiar issue when using the Regex type “MatchCollection” in .NET Core. Or to be more specific, in a .NET Standard library. And while the fix was simple, it actually was an interesting insight into .NET Core vs .NET Framework, and how the mini-fragmentation we have going on between the two frameworks is hard to document.

First let’s rewind. I started off writing code that was pretty similar to the following :

Regex.Matches(stringText, @"myRegex").Select(x => x.value);

When I wrote this code, I had written it inside a .NET Core project. Pretty simple and for the most part, the Regex part looks exactly the same as it did in .NET Framework. However I then had a need to move this code into a library to be shared across two different projects. Naturally I made the library .NET Standard and suddenly I got the following error.

‘MatchCollection’ does not contain a definition for ‘Select’ and no accessible extension method ‘Select’ accepting a first argument of type ‘MatchCollection’ could be found

Interesting. Same code, different platform (.NET Core vs .NET Standard), and suddenly our innocuous “framework” looking code doesn’t work. What gives?

Well, Here’s the definition of MatchCollection in .NET Core

public class MatchCollection : IList<Match>, IReadOnlyList<Match>, IList

And here’s the definition in .NET Framework.

public class MatchCollection : ICollection

So the .NET Core implementation actually implements more interfaces than that of .NET Framework. Infact because it implements IList<T>, it actually implements IEnumerable<T>. “Select”, is a LINQ method that works with IEnumerable<T>, but not ICollection. So using this in .NET Framework will not work.

But I’m using .NET Standard you say! That’s true but remember, .NET Standard enforces a “minimum” implementation – much like an interface would enforce a minimum requirement on a class. But it doesn’t mean that class can’t go over and beyond. As it so happens, the .NET Standard definition of MatchCollection only requires the implementation of ICollection, but it does not require IEnumerable<T> to be implemented. So you cannot use LINQ methods on a Regex MatchCollection if you are targeting anything other then .NET Core.

I kind of lived with this since it wasn’t such a big deal. But given I couldn’t use IEnumerable<T> methods on my MatchCollection anymore, I headed to the Microsoft Documentation for a couple of examples of how my could should work.

And here’s the actual issue.

The documentation *does not* show the differences between .NET Core and .NET Framework. The .NET Core 3.1 documentation for MatchCollection is found here : https://docs.microsoft.com/en-us/dotnet/api/system.text.regularexpressions.matchcollection?view=netcore-3.1

Note that it’s definition as per that documentation is :

public class MatchCollection : ICollection

That is wrong!

As it turns out, in 2018, this issue was raised here : https://github.com/dotnet/docs/issues/5608 which if you follow the trail of tickets, leads you to this ticket : https://github.com/MicrosoftDocs/feedback/issues/226. And right there slapped on the bottom.

Welp. Spamming Autocomplete options instead of reading documentation it is!

In C#, there is a grand total of 6 ways to concatenate a string. Those are :

• Using the + (plus) sign (Including +=)
• String.Concat
• String.Join
• StringBuilder
• String.Format
• Using String Interpolation (e.x. $”My string {variable}”).

I recently got asked about performance considerations when joining two strings together. I think everyone knows by now that using the + to join up large strings is (supposedly) a no no. But it got me thinking what actually are the performance implications? If you have two strings you want to concatenate, is it actually worth spinning up an instance of StringBuilder?

I wanted to do some quick benchmarking but by the end of the post, I ended up digging into the source code to atleast begin answering “why” things perform differently.

“Your Methodology Is Wrong!”

I don’t think I’ve ever written a benchmarking post without someone jumping on Twitter, Reddit, or some social media and pointing out how wrong I am. The thing is with benchmarking, and especially C#, there is so much “compiler magic” that happens. Things get optimized out or the compiler knows you are dumb and tries to help you out in a way you never expect.

If I’ve made a misstep somewhere, please drop a comment (Hell, plug your soundcloud while you’re at it). I always come back and add in comments where people think I’ve gone wrong and redo tests where needed. Sharing is caring after all!

My Setup

So as always, your mileage may vary when running these benchmarks yourself (but please do!). I am using an AMD Ryzen CPU with the .NET Core SDK as my runtime. Full details here :

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
AMD Ryzen 7 2700X, 1 CPU, 16 logical and 8 physical cores
.NET Core SDK=3.1.100
  [Host]     : .NET Core 3.1.1 (CoreCLR 4.700.19.60701, CoreFX 4.700.19.60801), X64 RyuJIT
  DefaultJob : .NET Core 3.1.1 (CoreCLR 4.700.19.60701, CoreFX 4.700.19.60801), X64 RyuJIT

Initial Benchmarking

For my benchmark, I’m going to try and do “single line joins”. What I mean by “single line joins” is that I have say 5 variables that I want to all join up in a long string, with a single space between them. I’m not doing this inside a loop and I have all 5 variables on hand. For this, I’m using BenchmarkDotNet.  My benchmark looks like so :

public class SingleLineJoin
{
    public string string1 = "a";
    public string string2 = "b";
    public string string3 = "c";
    public string string4 = "d";
    public string string5 = "e";

    [Benchmark]
    public string Interpolation()
    {
        return $"{string1} {string2} {string3} {string4} {string5}";
    }

    [Benchmark]
    public string PlusOperator()
    {
        return string1 + " " + string2 + " " + string3 + " " + string4 + " " + string5;
    }

    [Benchmark]
    public string StringConcatenate()
    {
        return string.Concat(string1, " ", string2, " ", string3, " ", string4, " ", string5);
    }

    [Benchmark]
    public string StringJoin()
    {
        return string.Join(" ", string1, string2, string3, string4, string5);
    }

    [Benchmark]
    public string StringFormat()
    {
        return string.Format("{0} {1} {2} {3} {4}", string1, string2, string3, string4, string5);
    }

    [Benchmark]
    public string StringBuilderAppend()
    {
        StringBuilder builder = new StringBuilder();
        builder.Append(string1);
        builder.Append(" ");
        builder.Append(string2);
        builder.Append(" ");
        builder.Append(string3);
        builder.Append(" ");
        builder.Append(string4);
        builder.Append(" ");
        builder.Append(string5);
        return builder.ToString();
    }
}

I’d also note that StringBuilder also has methods to do things like builder.AppendJoin which is like a hybrid between appending a line to the StringBuilder object but using a string.Join to actually create the line. I’ve skipped these because if you were simply going to use the AppendJoin method, you would instead just use string.Join anyway.

And the results are here :

Here’s the interesting thing for me. From what I can see, Interpolation, PlusOperator and Concat are roughly the same. String.Join is fast(er) with StringBuilder being the clear leader. String.Format is slowest by a mile. What’s going on here? We are going to have to do digging as to what goes on under the hood.

Digging Deeper

String.Format

Why is String.Format so slow? Well as it turns out, String.Format also uses StringBuilder behind the scenes, but it falls down to a method called “AppendFormatHelper” https://github.com/microsoft/referencesource/blob/master/mscorlib/system/text/stringbuilder.cs#L1322. Now this somewhat makes sense because you have to remember, string.Format can do things like :

String.Format("Price : {0:C2}", 14.00M);//Prints $14.00 (Formats as currency). 

So it has to do far more work in trying to format the string taking into account things like formatting a currency correctly etc. Even checking for these format types takes that little bit of extra time.

String.Join

String.Join is an interesting one because the code behind the scenes in my mind doesn’t make too much sense. If you pass in an IEnumerable or a params list of objects, then it simply uses a StringBuilder and doesn’t do much else : https://github.com/microsoft/referencesource/blob/master/mscorlib/system/string.cs#L161

But if you pass in params of string, it uses a char array and does some pretty low level stuff : https://github.com/microsoft/referencesource/blob/master/mscorlib/system/string.cs#L204

So immediately I think… Is there a difference? Well with this benchmark :

public class StringJoinComparison
{
    public string string1 = "a";
    public string string2 = "b";
    public string string3 = "c";
    public string string4 = "d";
    public string string5 = "e";

    public List<string> stringList;

    [GlobalSetup]
    public void Setup()
    {
        stringList = new List<string> { string1, string2, string3, string4, string5 };
    }


    [Benchmark]
    public string StringJoin()
    {
        return string.Join(" ", string1, string2, string3, string4, string5);
    }


    [Benchmark]
    public string StringJoinList()
    {
        return string.Join(" ", stringList);
    }
}

And the results :

Big difference. Infact it’s much much slower. Every now and again when I write benchmarks here, the original creator shows up and explains either A. Why I’m doing it wrong. Or B. Why it has to be this way, even with a performance hit. I would love to know what’s going on here because this one has almost a 2x difference depending on the input. Obviously there is different code behind the scenes, but it’s like a minefield here. I don’t think anyone would have suspected this.

String.Concat

Concat is very similar to Join. For example if we pass in an IEnumerable, it uses a StringBuilder : https://github.com/microsoft/referencesource/blob/master/mscorlib/system/string.cs#L3145

But if we pass in a params list of string, it instead falls down to the method ConcatArray : https://github.com/microsoft/referencesource/blob/master/mscorlib/system/string.cs#L3292

You may start noticing that a lot of methods have a call to “FastAllocateString”. Inferring from the usage and not from special knowledge that I have, it would appear that this allocates memory for the full size of the string, that is then “filled” up later on. For example given a list of strings, you already know ahead of time how large that string will be, so you can pre-allocate that memory and then simply fill in the bytes later.

Plus Operator

This one confused me a bit. I’m pretty sure from the moment I started programming in C#, I got told not to concat strings using the plus operator. But here it wasn’t so bad… Unfortunately I tried to find the source code like I’ve done above but to no avail. So I had to go on instinct to try and diagnose the issue.. Immediately I think I found it.

My hunch was that doing the operator in one big line was optimized out. So I wrote a small benchmark to test this theory :

[MemoryDiagnoser]
public class OperatorTest
{
    public string string1 = "a";
    public string string2 = "b";
    public string string3 = "c";
    public string string4 = "d";
    public string string5 = "e";


    [Benchmark]
    public string PlusOperatorWithResult()
    {
        var result = string1 + " ";
        result += string2 + " ";
        result += string3 + " ";
        result += string4 + " ";
        result += string5 + " ";
        return result;
    }


    [Benchmark]
    public string PlusOperator()
    {
        var result = string1 + " " + string2 + " " + string3 + " " + string4 + " " + string5;
        return result;
    }
}

If I’m being honest, I think there could still be some optimizer shenanigans going on here. But the idea is that with each string concat being on it’s own line, in theory it should have to create a new string each time. And the results :

So, a little bit of a slow down which is expected, but maybe not as much as I was expecting. Obviously over time, with larger strings and more joins, this could become more problematic which I think is what people try and point out when they scream “use StringBuilder for everything!”.

Also notice that I added the MemoryDiagnoser to this benchmark to show that yes, more memory is allocated when you are mucking using the += operator as it has to create a brand new string in memory to handle this.

StringBuilder

StringBuilder’s source code can be found here : https://github.com/microsoft/referencesource/blob/master/mscorlib/system/text/stringbuilder.cs. It’s relatively simply in that it holds a char array until the final moment and then joins everything up right at the end. The reason it’s so fast is because you are not allocating strings until you really need it.

What surprised me most about the use of StringBuilder is that even at 5 appends (Or I guess more if we count the spaces), it’s much much faster than just using the + operator. I thought there would be some sort of breakpoint in the tens, maybe hundreds of concats that the overhead of a StringBuilder becomes more viable. But it seems “worth it”, even if you are only doing a few concats (but more on that below).

Interpolation

I actually can’t find the source code for what does Interpolation in C#. Infact I wasn’t even sure what to search. Because it’s similar to the plus operator, I assume that it’s maybe just sugar around the same piece of code that joins strings deep in the code.

Summary

So where does that leave us? Well we came in with the knowledge that StringBuilder was best practice for building strings, and we left with that still intact. We also found that even when building smaller strings, StringBuilder out performs the rest. Does that mean immediately rewrite your code to use StringBuilder everywhere? I personally doubt it. Based on readability alone, a few nano seconds might not be worth it for you.

We also walk away with the knowledge that string.Format performs extremely poorly even when we aren’t doing any special formatting. Infact we could use literally any other method to join strings together and have it be faster.

And finally, we also found that string concatenation is still a strange beast. With things like string.Concat and string.Join doing every different things depending on what you pass in. 9 times out of 10 you probably don’t even think there is a difference between passing in a IEnumerable vs Params, but there is.

A graduate developer asked a simple question the other day.

Grad : “What type should I return from this method?”
Me : “Make it a list”

Seemed simple enough. I took a look at his code and was confounded on just what had gone wrong. Instead of using List<T>, they had used the type “ArrayList”. I honestly can’t even remember the last time I used ArrayList. I think maybe right when I started programming in .NET 2, I couldn’t understand generics quick enough and the ArrayList seemed like a drop in replacement. It’s not!

What’s The Difference?

The key difference between the two is that an ArrayList holds only types of “objects”. That means theoretically it’s a box of anything you want it to be. For example this code compiles just fine :

ArrayList arrayList = new ArrayList();
arrayList.Add(123);
arrayList.Add("abc");
arrayList.Add(new object());

It’s then on the code grabbing things out of the array list to “check” that it’s the correct type. In practice it’s not going to be so haphazard that you are throwing all sorts of types in an array list so really it’s more of a compile time “looseness”. If we compare it to a List :

List<int> list = new List<int>();
list.Add(123);
list.Add("abc"); //Compile time error

No bueno. It knows that we only want to be storing integers and trying to jam anything else in there isn’t going to fly.

But what about this?

List<object> list = new List<object>();
list.Add(123);
list.Add("abc");

That works right? A list of objects is almost the same thing as an ArrayList. Almost.

If we look at the interfaces implemented by ArrayList :

public class ArrayList : ICollection, IEnumerable, IList, ICloneable

List is basically the same with a few generic interfaces thrown in. However when you check these, they don’t add anything except generic methods of things like “Add” etc that use the type :

public class List<T> : ICollection<T>, IEnumerable<T>, IEnumerable, IList<T>, IReadOnlyCollection<T>, IReadOnlyList<T>, ICollection, IList

But where things change is using LINQ. Almost all methods (I say almost but I think it’s all) are built upon IEnumerable<T> and not IEnumerable. For example the Where clause in LINQ looks like :

public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source, Func<TSource, bool> predicate)

That means you cannot use LINQ on an ArrayList. In some use cases it’s no biggie, but you do get LINQ for free… So picking a type that doesn’t support it is really shooting yourself in the foot for no reason.

Performance Implications

In some cases, there are large performance implications when picking an ArrayList over a List<T>. That comes down to the act of “boxing” and “unboxing”. In simple terms, boxing is taking a value type (such as an integer) and wrapping it in an object and storing it on the heap instead of the stack. Microsoft actually have a great article on the subject here : https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/types/boxing-and-unboxing

But how does that affect the List vs ArrayList conversation? When we store an item in an ArrayList it must be of type object (Or a type of). If we are storing a value type in our ArrayList, then before it can be stored, it must first “box” the object and wrap it. A List<int> does not have the same boxing cost (Although a List<object> would).

To test this, I created a benchmark using BenchmarkDotNet

public class ArrayListVsListWrite
{
    int itemCount = 10000000;
    public ArrayList arrayList;
    public List<int> list;
    public List<object> listObject;

    [IterationSetup]
    public void Setup()
    {
        arrayList = new ArrayList();
        list = new List<int>();
        listObject = new List<object>();
    }

    [Benchmark]
    public ArrayList WriteArrayList()
    {
        for(int i=0; i < itemCount; i++)
        {
            arrayList.Add(i);
        }
        return arrayList;
    }

    [Benchmark]
    public List<object> WriteListObject()
    {
        for (int i = 0; i < itemCount; i++)
        {
            listObject.Add(i);
        }
        return listObject;
    }

    [Benchmark]
    public List<int> WriteList()
    {
        for (int i = 0; i < itemCount; i++)
        {
            list.Add(i);
        }
        return list;
    }
}

In simple terms. We are looping 10 million times and adding the item to the list. The results of which are :

Not hard to see the performance difference here. We can see that a List of type object also suffers the same boxing/unboxing problem.

To a lesser extent, reading would also be slower as you would be reading an object, and then “unboxing” that object and casting it to an integer. I was going to do a benchmark with that but… you get the idea.

When Should You Use ArrayList?

Honestly. Never.

The only time you should use an ArrayList is when you are using a library built before List<T> which I believe was introduced in .NET 2.0. If you are using a library (Or building code) that targets .NET 1.0 or 1.1, then I guess ArrayList is OK. It’s probably going to be the least of your problems.

Are there new .NET Core API projects that don’t use Swagger? It’s one of the first things I install on a new project and even if I end up testing an API with something like Postman, the fact that I can quickly show a GUI to run any API endpoints for testers, other devs, hell even project managers is phenomenal.

Now I’ve actually written about using Swagger in .NET Core before, but with the release of .NET Core 3 came some real gotchas that caused some massive headaches. Suddenly my “let’s just spend 30 seconds adding Swagger” turned into a couple of hours trying to work out why things don’t work quite as seamless as before. I’ll write this as a quick walkthrough that points out the issues along the way. It should be enough to get you back up and running before too long!

Installing Swagger

The first thing is installing Swagger via Nuget. Now the annoying thing is that the actual package you want to install is not called Swagger at all. You actually want to install the Swashbuckle.AspNetCore package which then installs various swagger dependencies. Maybe this nuget browser screenshot will explain better than I can

The other frustrating thing (Although now fixed), is that upon release of .NET Core 3, the release version of Swashbuckle did not actually work with .NET Core 3 and you had to use a pre-release version. That’s hopefully fixed now, but just incase, ensure that you are using atleast version 5.0.0 of Swashbuckle. If you use a version before this you will get a really random error :

Swashbuckle.AspNetCore.SwaggerGen.SwaggerGenerator': Failed to compare two elements in the array

Again, the fix is to ensure you are running version 5.0.0 or above of Swashbuckle.

Adding Swagger Services

The next step is to add the Swagger services to the ServiceCollection in .NET Core. In your ConfigureServices method of startup.cs, you want to add a line that looks like this :

services.AddSwaggerGen(swagger =>
{
    swagger.SwaggerDoc("v1", new OpenApiInfo { Title = "My API" });
});

Now pay attention to the fact that I’m creating an object called “OpenApiInfo”, this used to be called just “Swagger.Info”. So if you are copy and pasting from a previous project or an old tutorial you’re gonna run into something like :

The type or namespace name 'Info' does not exist in the namespace 'Swashbuckle.AspNetCore.Swagger'

This goes for almost all configuration with Swagger there has been a rename to prepend “OpenApi” to almost all configuration properties. If you are upgrading a project, 9 times out of 10, you will just have to tack on the “OpenApi” part.

It’s an annoying thing to do but for the most part you can guess your way through it and have everything working.

Adding Swagger Middleware

The final step is to add our Swagger middleware to actually serve up our nice HTML interface. There are two calls to add to our Configure method in our startup.cs. These are :

app.UseSwagger();
app.UseSwaggerUI(c =>
{
    c.SwaggerEndpoint("/swagger/v1/swagger.json", "My API");
});

These should come *before* any call to UseMVC or similar that might try and serve the request before Swagger can get to it. So in practice, near the top.

The first call to UseSwagger is the one that adds the “JSON” serving capability to your app. The second call is what adds the actual HTML interface for Swagger. Nice and easy!

NewtonSoft.JSON Support

This one got me good and caused massive headaches. So by now we know that .NET Core 3+ will not use JSON.NET as the default JSON serializer. We can fix that with some easy code (post on that here!). But even once you change your API to use JSON.NET as the serializer, Swagger will still use the default System.Text.Json serializer anyway!

But of course, there is a nuget package for that! Run the following from your package manager console :

Install-Package Swashbuckle.AspNetCore.Newtonsoft

Then in your ConfigureServices method of your startup.cs, add the following :

services.AddSwaggerGenNewtonsoftSupport();

Now Swagger will also abide by JSON.NET serialization rules and will align with your API.

I’ve recently been trying out the new System.Text.Json JSON Parser that is now built into .NET Core 3+, replacing NewtonSoft.Json (Sometimes called JSON.NET) as the default JSON parser in ASP.NET Core. There has been quite a few gotchas and differences between the two libraries, but none more interesting than the following piece of documentation :

During deserialization, Newtonsoft.Json does case-insensitive property name matching by default. The System.Text.Json default is case-sensitive, which gives better performance since it’s doing an exact match.

I found this interesting, especially the last line which suggests that doing exact matches by default results in much better performance.

Interestingly enough, there is also the following piece of info :

If you’re using System.Text.Json indirectly by using ASP.NET Core, you don’t need to do anything to get behavior like Newtonsoft.Json. ASP.NET Core specifies the settings for camel-casing property names and case-insensitive matching when it uses System.Text.Json.

So that essentially means when we switch to System.Text.Json in an ASP.NET Core project specifically, things will be case insensitive by default, and by extension, have slightly worse performance than forcing things to be case sensitive. By the way, for the record, I think this should be the default behaviour because in 99% of Web SPA cases, the front end will be using javascript which typically is written using camelCase, and if the backend is in C#, the properties are typically written in PascalCase.

But here’s the thing I wondered. When everyone’s out there posting benchmarks, are they benchmarking case sensitive or case insensensitive JSON parsing? In another blog post I and many others often refer back to for JSON benchmarks (https://michaelscodingspot.com/the-battle-of-c-to-json-serializers-in-net-core-3/) they are actually doing the case sensitive parsing which again, I don’t think is going to be the reality for the majority of use cases.

Let’s dig a little more!

Benchmarking

The first thing I wanted to do was create a simple benchmark to test my hypothesis. That is, does deserializing data with case insensitivity turned on slow down the deserialization process.

I’m going to use BenchmarkDotNet for this purpose. The class I want to deserialize looks like so :

public class MyClass
{
    public int MyInteger { get; set; }

    public string MyString { get; set; }

    public List<string> MyList { get; set; }
}

Now I also had an inkling of a theory this wouldn’t be as simple as first thought. I had a hunch that possibly that even when case insensitivity was turned on, if it could find an exact match first, it would attempt to use that anyway. e.g. It’s possible that the code would look something like this pseudo code :

if(propertyName == jsonProperty)
{
    //We found the property on an exact match. 
}else if(propertyName.ToLower() == jsonProperty.ToLower())
{
   //We found it after ToLowering everything. 
}

With that in mind, I wanted my benchmark to test serializing both PascalCase names (So exact match), and camelCase names. My benchmark looked like so :

public class SystemTextVsJson
{
    private readonly JsonSerializerOptions options = new JsonSerializerOptions() { PropertyNameCaseInsensitive = true };

    private const string _jsonStringPascalCase = "{\"MyString\" : \"abc\", \"MyInteger\" : 123, \"MyList\" : [\"abc\", \"123\"]}";
    private const string _jsonStringCamelCase = "{\"myString\" : \"abc\", \"myInteger\" : 123, \"myList\" : [\"abc\", \"123\"]}";

    [Benchmark]
    public MyClass SystemTextCaseSensitive_Pascal()
    {
        return JsonSerializer.Deserialize<MyClass>(_jsonStringPascalCase);
    }

    [Benchmark]
    public MyClass SystemTextCaseInsensitive_Pascal()
    {
        return JsonSerializer.Deserialize<MyClass>(_jsonStringPascalCase, options);
    }

    [Benchmark]
    public MyClass SystemTextCaseSensitive_Camel()
    {
        return JsonSerializer.Deserialize<MyClass>(_jsonStringCamelCase);
    }

    [Benchmark]
    public MyClass SystemTextCaseInsensitive_Camel()
    {
        return JsonSerializer.Deserialize<MyClass>(_jsonStringCamelCase, options);
    }
}

And just so we are all on the same page, the machine I’m running this on looks like :

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
AMD Ryzen 7 2700X, 1 CPU, 16 logical and 8 physical cores
.NET Core SDK=3.1.100

Now onto our results :

Interesting. Very interesting. So a couple of things that stick out to me immediately.

If we are using PascalCase for our property names on both ends (in the JSON and our C# class), then the case sensitivity setting doesn’t matter all too much. This proves my initial thoughts that it may try for an exact match no matter the setting as that’s likely to be faster than any string manipulation technique.

Next. Slightly of interest is that when parsing with case sensitivity turned on, when there is no match (e.g. You have screwed up the casing on one of the ends), it runs slightly slower. Not by much. But enough to be seen in the results. This is probably because it tries to do some extra “matching” if it can’t find the exact match.

Finally. Oof. Just as we thought. When we are doing case insensitive matching and our incoming data is camelCase with the class being PascalCase, the benchmark is substantially slower than exact matching. And I just want to remind you, the default for ASP.NET Core applications is case insensitive.

So, how does this actually stack up?

Benchmarking Against Newtonsoft

The interesting thing here was if we are comparing apples to apples, Newtonsoft also does case insensitive matching but it does so by default. So when we do any benchmarking against it, we should try and do so using similar settings if those settings would be considered the norm.

With that in mind, let’s do this benchmark here :

public class SystemTextVsJson
{
    private readonly JsonSerializerOptions options = new JsonSerializerOptions() { PropertyNameCaseInsensitive = true };

    private const string _jsonStringCamelCase = "{\"myString\" : \"abc\", \"myInteger\" : 123, \"myList\" : [\"abc\", \"123\"]}";

    [Benchmark]
    public MyClass SystemTextCaseInsensitive_Camel()
    {
        return JsonSerializer.Deserialize<MyClass>(_jsonStringCamelCase, options);
    }

    [Benchmark]
    public MyClass NewtonSoftJson_Camel()
    {
        return Newtonsoft.Json.JsonConvert.DeserializeObject<MyClass>(_jsonStringCamelCase);
    }
}

And the results?

Much much much closer. So now that we are comparing things on an even footing the performance of System.Text.Json to NewtonSoft actually isn’t that much better in terms of raw speed. But what about memory? I hear that touted a lot with the new parser.

Memory Footprint

BenchmarkDotNet gives us the ability to also profile memory. For our test, I’m going to keep the same benchmarking class but just add the MemoryDiagnoser attribute onto it.

[MemoryDiagnoser]
public class SystemTextVsJson
{
}

And the results

Wow, credit where credit is due, that’s a very impressive drop. Now again I’m only testing with a very minimal JSON string, but I’m just looking to do a comparison between the two anyway.

Final Thoughts

Why did I make this post in the first place? Was it to crap all over System.Text.Json and be team JSON.NET all the way? Not at all. But I have to admit, there is some level of frustration when moving to using System.Text.Json when it doesn’t have the “features” that you are used to in JSON.NET, but it’s touted as being much faster. Then when you dig a little more in the majority of use cases (case insensitive), it’s not actually that much faster.

And I have to point out as well. That literally everytime I’ve written a benchmarking post for C# code, I’ve managed to get something wrong where I didn’t know the compiler would optimize things out etc and someone jumps in the Reddit comments to call me an idiot (Will probably happen with this one too! Feel free to drop a comment below!). So you can’t really blame people doing benchmarks across JSON parsers without realizing the implications of casing because the fact ASP.NET Core has specific defaults that hide away this fact means that you are unlikely to run into the issue all that often.

If you are in the same boat as me and trying to make the leap to System.Text.Json (Just so you stay up to date with what’s going on), I have a post sitting in my drafts around gotchas with the move. Case sensitivity is a big one but also a bunch of stuff on various defaults, custom converters, null handling etc which were all so great in JSON.NET and maybe a little less great in System.Text.Json. So watch this space!