For the past few years, everytime I’ve started a new project there has been one sure fire class that I will copy and paste in on the first day. That has been my “TestingContext”. It’s sort of this one class unit testing helper that I can’t do without. Today, I’m going to go into a bit of detail about what it is, and why I think it’s so damn awesome.

First off, let’s start about what I think beginners get wrong about unit testing, and what veterans begin to hate about unit testing.

The Problem

The number one mistake I see junior developers making when writing unit tests is that they go “outside the scope” of the class they are testing. That is, if they are testing one particular class, let’s say ServiceA, and that has a method that calls ServiceB. Your test actually should never ever enter ServiceB (There is always exceptions, but very very rare). You are testing the logic for ServiceA, so why should it go and actually run code and logic for ServiceB. Furthermore, if your tests are written for ServiceA, and ServiceB’s logic is changed, will that affect any of your tests? The answer should be no, but it often isn’t the case. So my first goal was :

Any testing helper should limit the testing scope to a single class only.

A common annoyance with unit tests is that when a constructor changes, all unit tests are probably going to go bust even if the new parameter has nothing to do with that particular unit test. I’ve seen people argue that if a constructor argument changes, that all unit tests should have to change, otherwise the service itself is obviously doing too many things. I really disagree with this. Unless you are writing pretty close to one method per class, there is always going to be times where a new service or repository is injected into a constructor that doesn’t really change anything about the existing code. If anything, sticking to SOLID principles, the class is open for extension but not modification. So the next goal was :

Changes to a constructor of a class should not require editing all tests for that class.

Next, when writing tests, you should try and limit the amount of boilerplate code in the test class itself. It’s pretty common to see a whole heap of Mock instantiations clogging up a test setup class. So much so it becomes hard to see exactly what is boilerplate and going through the motions, and what is important setup code that needs to be there. On top of that, as the class changes, I’ll often find old test classes where half of the private class level variables are assigned, but are actually never used in tests as the class was modified. So finally :

Boilerplate code within the test class should be kept to a minimum.

Building The Solution

I could explain the hell out of why I did what I did and the iterations I went through to get here, but let’s just see the code.

First, you’re gonna need to install the following Nuget package:

This actually does most of the work for us. It’s an auto mocking package that means you don’t have to create mocks for every single constructor variable regardless of whether it’s used or not. You can read more about the specific package here : https://github.com/AutoFixture/AutoFixture. On it’s own it gets us pretty close to solving our problem set, but it doesn’t get us all the way there. For that we need just a tiny bit of a wrapper.

And that wrapper code :

Essentially it’s an abstract class that your test class can inherit from, that provides a simple way to abstract away everything about mocking. It means our tests require minimal boilerplate code, and rarely has to change based on class extension. But let’s take an actual look how this thing goes.

Testing Context In Action

To show you how the testing context works, we’ll create a quick couple of test classes.

First we just have a repository that returns names, then we have a service that has a couple of methods on it that interact with the repository, or in some cases a utility class.

Now two things here, the first being that the TestService takes in an ITestRepository (Interface) and UtilityService (class), so this could get a bit gnarly under normal circumstances because you can’t mock a concrete class. And second, the first method in the service, “GetNamesExceptJohn” doesn’t actually use this UtilityService at all. So I don’t want to have to mess about injecting in the class when it’s not going to be used at all. I would normally say you should always try and inject an interface, but in some cases if you are using a third party library that isn’t possible. So it’s more here as an example of how to get around that problem.

Now onto the tests. Our first test looks like so :

The first thing you’ll notice that we inherit from our TestingContext, and pass in exactly what class we are going to be testing. This means that it feels intuitive that the only thing we are going to be writing tests for is this single class. While it’s not impossible to run methods from other classes in here, it sort of acts as blinders to keep you focused on one single thing.

Our test setup calls the base.Setup() method which just preps up our testing context. More so, it clears out all the data from previous tests so everything is stand alone.

And finally, our actual test. We simply ask the context to get a mock for a particular interface. In the background it’s either going to return one that we created earlier (More on that shortly), or it will return a brand new one for us to setup. Then we run “ClassUnderTest” with the method we are looking to test, and that’s it! In the background it takes any mocks we have requested, and creates an instance of our class for us. We don’t have to run any constructors at all! How easy is that.

Let’s take a look at another test :

In this test, we are doing everything pretty similar, but instead are injecting in an actual class. Again, this is not a good way to do things, you should try and be mocking as much as possible, but there are two reasons that you may have to inject a concrete class.

  1. You want to write a fake for this class instead of mocking it.
  2. You are using a library that doesn’t have an interface and so you are forced to inject the concrete class.

I think for that second one, this is more a stop gap measure. I’ve dabbled taking out the ability to inject in classes, but there is always been atleast one test per project that just needed that little bit of extensibility so I left it in.

The Niceties

There is a couple of really nice things about the context that I haven’t really pointed out too much yet, so I’ll just highlight them here.

Re-using Mocks

Under the hood, the context keeps a list of mocks you’ve already generated. This means you can reuse them without having to have private variables fly all over the place. You might have had code that looked a bit like this in the past :

You can now rewrite like :

This really comes into it’s own when your setup method typically contained a huge list of mocks that you setup at the start, then you would set a class level variable to be re-used in a method. Now you don’t have to do that at all. If you get a mock in a setup method, you can request that mock again in the actual test method.

Constructor Changes Don’t Affect Tests

You might have seen on the first test we wrote above, even though the constructor required 2 arguments, we only bothered mocking the one thing we cared about for the method under test. Everything else we can let the fixture handle.

How often do you see things like this in your tests?

And then someone comes along and adds a new constructor argument, and you just throw a null onto the end again? It’s a huge pain point of mine and makes tests almost unreadable at times.

Test Framework Agnostic

While in the above tests I used NUnit to write my tests, the context itself doesn’t require any particular testing framework. It can work with NUnit, MSTest and XUnit.

Revisiting Our Problems

Let’s go full circle, and revisit the problems that I found with Unit Testing.

Any testing helper should limit the testing scope to a single class only.

I think we covered this pretty well! Because we pass the class we are looking to test into our testing context, it basically abstracts away being able to call other classes.

Changes to a constructor of a class should not require editing all tests for that class.

We definitely have this. Changes to the constructor don’t affect our test, and we don’t have to setup mocks for things we are uninterested in.

Boilerplate code within the test class should be kept to a minimum.

This is definitely here. No more 50 line setup methods just setting up mocks incase we need them later. We only setup what we need, and the re-usability of mocks means that we don’t even need a bag full of private variables to hold our mock instances.

What’s Your Thoughts?

Drop a comment below with your thoughts on the testing context!

I want to start off this post by saying if you are starting a new .NET Core project and you are looking to use a ServiceLocator. Don’t. There are numerous posts out there discussing how using a ServiceLocator is an “anti-pattern” and what not, and frankly I find anything that uses a ServiceLocator a right pain in the ass to test. Realistically in a brand new .NET Core project, you have dependency injection out of the box without the need to use any sort of static ServiceLocator. It’s simply not needed.

But, if you are trying to port across some existing code that already uses a ServiceLocator, it may not be as easy to wave a magic wand across it all and make everything work within the confines of .NET Core’s dependency injection model. And for that, we will have to work out a new way to “shim” the ServiceLocator in.

An important thing to note is that the “existing” code I refer to in this post is the “ServiceLocator” class inside the “Microsoft.Practices” library. Which itself is also part of the “Enterprise Library”. It’s a little confusing because this library is then dragged along with DI frameworks like Unity back in the day, so it’s hard to pinpoint exactly what ServiceLocator you are using. The easiest way is, are you calling something that looks like this :

If the answer is yes, then you are 99% likely using the Microsoft.Practices ServiceLocator. If you are using a different service locator but it’s still a static class, you can probably still follow along but change the method signature to your needs.

Creating Our Service Locator Shim

The first thing we are going to do is create a class that simply matches our existing ServiceLocator structure and method signatures. We want to create it so it’s essentially a drop in for our existing ServiceLocator so all the method names and properties should match up perfectly. The class looks like :

It can be a bit confusing because we are mixing in static methods with instance ones. But let’s walk through it.

On the static end, we only have one method that is SetLocatorProvider , this allows us to pass in a ServiceProvider  instance that will be used for all service location requests. ServiceProvider is the built in DI that comes with .NET Core (We’ll take a look at how we hook it up in a second). We also have a static property called Current  that simply creates an actual instance of ServiceLocator, providing us with access to the “instance” methods.

Once we have an instance of the ServiceLocator class, we then gain access to the GetInstance  methods, which perfectly match the existing ones of the old ServiceLocator class. Awesome!

Wiring It Up To .NET Core Service Provider

The next part is easy! In our ConfigureServices method of our startup.cs. We need to set the LocatorProvider. It looks like so :

So all we are doing is passing in an instance of our ServiceProvider and this will be used to fetch any instances that are required going forward.

This is actually all we need to do. If you have existing code that utilizes the ServiceLocator, barring a change to any “Using” statements to be swapped over, you should actually be all ready to go!

Testing It Out

Let’s give things a quick test to make sure it’s all working as intended.

I’m going to create a simple test class with a matching interface.

I need to wire this up in my ConfigureServices method of startup.cs

Then I’m just going to test everything on a simple API endpoint in my .NET Core web app.

Give it a run and…

All up and running!

I’m currently living the whole snake case vs camel case argument all over again. Me being a web developer, I prefer camel case (myVariable) as it fits nicely with any javascript code. Others, in what seems predominately iOS developers, prefer to use snake case (my_variable). Typically it’s going to be personal preference and on any internal API, you can do whatever floats your boat. But what about for public API’s? Can we find a way in which we can let the consumer of the API decide how they want the API to return payloads? Why yes, yes we can.

Serialization Settings In .NET Core

But first, let’s look at how we can control JSON serialization in .NET Core if we wanted to go one way over another.

We can override any particular property on a model to say always be serialized with a particular name. This is really heavy handed and I think is probably the worse case scenario. When you do this, it’s basically telling the global settings to get stuffed and you know better.

So for example if I have a model like so :

It will be serialized like :

So you’ll notice that the first property has the first letter capitalized (essentially PascalCase), but the second property without the JsonProperty attribute is using the default .NET Core serialization which is camelCase.

This may seem OK, but what if you then change the default serialization for .NET Core? What’s it going to look like then?

Let’s head over to our ConfigureServices method inside our Startup.cs. There we can change a couple of settings on our “AddMvc” call. Let’s say we now want to go with Snakecase for everything, so we change our JsonOptions to the following :

Running this, we now get :

Well… it did what we told it to do, but it’s definitely done us over a little bit. The property that we have overridden the naming of has won out against our “default” naming strategy. It’s what we expect to see, but often before people realize this it’s too late to go all the way back through the code and change it.

This will often become a problem when you’ve renamed a property not because of any naming strategy dogma, but because you actually want it to be called something else. So imagine the following :

You’ve renamed it not because you wanted to override the naming strategy, but because you literally wanted it to be serialized under a different name. Usually because a name that makes sense internally may not make total sense to the outside world.

At this point there is no getting around this, you can make a completely custom attribute and have that handled in a custom contract resolver (Which we will investigate shortly!), but there is nothing inside the framework to help you be able to rename a property that will still respect the overall naming strategy.

Dynamic Serialization At Runtime

So far we have seen how to hard code a particular naming strategy at compile time, but nothing that can be dynamically changed. So let’s work that out!

Within JSON.net, there isn’t any ability for this to do it out of the box. In my case I’m looking to use a naming strategy based on a particular header being passed through an API. Without there being one, I had to roll my own. Now the next part of this will be sort of like a brain dump of how I got up and running. And to be fair, I worked this as a proof of concept on a small API I was working on, so it may not be great at scale, but it’s a great starting point.

So to begin with, we have to create a custom “naming” strategy that inherits from JSON.net’s abstract class “NamingStrategy”. Inside this we need to override the method called “ResolvePropertyName” to instead return the name we want to use based on an API header. Phew. OK, here’s how I went about that :

OK so it’s kinda big, so let’s break it down into chunks.

First we have our settings class :

This contains a HeaderName for what the header is going to be called, a dictionary called NamingStrategies where the keys are the header values we might expect, and their corresponding naming strategies (More on this later). We also have our DefaultStrategy incase someone doesn’t pass a header at all. Next we have a func that will return an HttpContextAccessor. We need this because in .NET Core, HttpContext is no longer an application wide static property available everywhere, it actually needs to be injected. Because we aren’t actually using DI here, we instead need to pass in a “function” that will return an HttpContextAccessor. We’ll delve more into this when we get to our configuration.

The rest of the code should be pretty straight forward. We get the header, we check if it’s valid (Matches anything in our dictionary), and if it does, we use that strategy to get the property name. If it doesn’t we then just use the default naming strategy.

Now, at this point I actually thought I was done. But as it turns out, JSON.net has aggressive caching so it doesn’t have to work out how to serialize that particular type every single request. From what I’ve seen so far, this is more about the actual custom serialization of the values, not the names, but the naming sort of gets caught up in it all anyway. The caching itself is done in what’s called a “ContractResolver”. Usually you end up using the “DefaultContractResolver” 99% of the time, but you can actually create your own and within that, setup your own caching.

Here’s mine that I created to try and overcome this caching issue:

So what does this actually do? Well because we inherited from the DefaultContractResolver, for the most part it actually does everything the same. With some key differences, let’s start at the top.

When we construct the resolver, we pass in our naming strategy options (Truth be told, I’m not sure I like it this way, but I wasn’t originally intending to have to do a resolver so the options are for the “namingstrategy” not the resolver. Bleh). And we also pass in a “function” that can return a memory cache, again in .NET Core, memory caches are not side wide. JSON.NET actually just uses a static dictionary which also seemed OK, but I like MemoryCache’s wrappers a bit more.

The only thing we override is the ResolveContract method which is where it’s doing the aggressive caching. We actually want to cache things too! But instead of caching based purely on the type (Which is what the default does), we want to also find what the header was that was passed in. This way we cache for the combination of both the header value and the type. To get the header name, I actually reach out to the naming strategy which I probably shouldn’t be doing, but it was an easy way to “share” the logic of getting a valid header

Now it’s time to set everything up. Here’s how it looks inside our ConfigureServices method of our startup.cs :

A bit of a dogs breakfast but it works. We set up the contract resolver to use our new ApiHeaderJsonContractResolver.  We pass in our naming strategy options,  saying that the default should be CamelCase, the header should be “json-naming-strategy”, for our HttpContextAccessorProvider we tell it to use our ServiceCollection to get the service, we pass in a list of valid naming strategies. Finally we also pass in our function that should be used to pull the memory cache.

Let’s test it out!

First let’s try calling it with no header at all :

Cool, it used our default camelcase naming strategy.

Let’s tell it to use snake case now!

Perfect!

And forcing things back to camel case again!

Awesome!

So here we have managed to create our own resolver and naming strategy to allow clients to specify which naming convention they want. To be honest, it’s still a work of progress and this thing is definitely at the proof of concept stage, I’m still getting to grips on the internal workings of JSON.net, but it’s definitely a good start and OK to use on any small project you have going!

I’ve seen some fierce office arguments about how to use HttpClient  in .NET since I’ve been programming. And it’s always about one thing. When exactly do you dispose of an HttpClient instance?

You see there is one train of thought that looks like this :

So you are creating a new instance every time you make an outbound call. Certainly when I first started using the HttpClient class, this seemed logical. But within the past couple of years, this particular article has become pretty infamous : https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/. With the key quotes being :

If we share a single instance of HttpClient then we can reduce the waste of sockets by reusing them

and

In the production scenario I had the number of sockets was averaging around 4000, and at peak would exceed 5000, effectively crushing the available resources on the server, which then caused services to fall over. After implementing the change, the sockets in use dropped from an average of more than 4000 to being consistently less than 400, and usually around 100.

Pretty damning stuff. So in this example, it’s instead favoured to re-use HttpClient instances across the application. And I have to admit, before this article was sent my way, I was definitely in the “always wrap it in a using statement” camp, and that’s generally all I saw out in the wild. These days it’s gone completely the other way, and you would now expect a “static” instance of HttpClient to be created and reused for the lifetime of the application. (There is actually now articles telling you to *not* use a single instance!)

But of course, in comes .NET Core with a new way to manage HttpClient lifetimes, and it’s an interesting one! This guide revolves around using .NET Core 2.1. If you aren’t using version 2.1 yet, there is a handy guide here to get up and running.

HttpClient Factories

Because we are working with .NET Core, and Core has fallen in love with “Dependency Inject all the things”! Then of course Microsoft’s solution for the HttpClient messiness is a DI solution. Let’s imagine that I’m creating an API wrapper for Twitter. So I’m going to create a “TwitterApiClient” class to encapsulate all of this work.

For the sake of brevity, I’m not actually calling out to the Twitter API. But you get the idea that I’ve created a nice wrapper for the API, that has a method called “GetTweets” that would if I wanted to, reach out and get some tweets and return them as a list. Let’s just use our imagination here! You’ll also notice I did this the crap way where we wrap everything in a using statement. This is intentional for now, just to show how things “might have been” before we knew better!

In my ConfigureServices method I’m going to register my TwitterApiClient like so :

Now I’m going to go ahead and create a controller that just gets these tweets and writes them on the screen :

Run this bad boy and what do we see?

OK so in theory we have everything working, but it’s disposing of our HttpClient each time which as we know from the above article, is a bad idea. So we could create a static instance of HttpClient, but to be perfectly honest, I hate static instances. But we are cutting edge so we are using .NET Core 2.1 (Again if you aren’t yet, you can read our guide to getting up and running with 2.1 here), and now we can use an injected instance of HttpClient. So let’s do that!

First we change our class around. We instead inject in an instance of HttpClient and use this instead.

Now if you run this at this point, you are gonna see an error close to :

InvalidOperationException: Unable to resolve service for type ‘System.Net.Http.HttpClient’ while attempting to activate […]

This is because Core doesn’t just inject in HttpClient’s by default, there is a tiny bit of configuration needed.

First, we need to install the Microsoft.Extensions.Http nuget package. At the time of writing this is in preview so you will need the full version install command. So from your package manager console it will be something like:

Now back in our ConfigureServices method in our startup.cs. We are going to add a call to AddHttpClient like so but most importantly, we remove our original call to add a transient instance of our original client. This is super important. I banged my head against a wall for a long time trying to work out what was going wrong with my code. And it turns out when you call AddHttpClient, it actually does a bunch of wiring up for you. If you then call AddTransient yourself, you just overwrite the lot!

Give it a run and we should now be all up and running! Now what this code actually does is tell .NET Core to inject in an HttpClient instance into your nice little API wrapper, and it will handle the lifetimes for it. That last part is important. It’s not going to be a singleton, but it’s not going to be a per request type thing either. .NET Core has magic sauce under the hood that means it will at times recycle the underlying connections when it thinks it should.

I’ll admit it’s sort of hazy in a way that Microsoft says “trust us. We’ll sort this for you”. But it’s probably a whole lot better than what you were doing on your own.

Setting Defaults

Taking things a step further, we can actually set up some defaults in our configure method that mean we are configuring our application all in one place (And it’s not hardcoded in our services). As an example, I can do this :

If we decide to read these settings from a config store like appSettings.json, we don’t have to pollute any sort of IOptions throughout our actual Twitter client. Nice!

Named HttpClient Factories

Something else you can do is create named instances of HttpClient that can be created from an HttpClientFactory. This is handy if the class you want to inject HttpClient instances into needs more than one default. So for example a “SocialMediaApiClient” that talks to both Twitter and Facebook.

The setup is slightly different. Instead of saying which class we want to inject our HttpClient into, we just add an instance of HttpClient to the factory with a particular name, and the defaults we want.

Then when it comes to our actual service we first inject an instance of IHttpClientFactory, and then we can get that specific instance of HttpClient by calling CreateClient with the client name as a parameter. Once again, the lifecycle is managed for us as to when it’s disposed of or reused.

Generic HttpClient

Finally, the HttpClient factory comes with the ability to generate a new HttpClient on demand which will be managed for you. With this, there shouldn’t ever be a reason to “new up” an instance of HttpClient ever again.

First we just call “AddHttpClient” in our ConfigureServices method, passing in absolutely nothing.

And whenever we want to actually get a new instance of an HttpClient. We inject in an instance of IHttpClientFactory and call CreateClient passing in nothing extra.

 

While working on another post about implementing your own custom ILogger implementation, I came across a method on the ILogger interface that I had never actually used before. It looks something like  _logger.BeginScope("Message Here");  , Or correction, it didn’t just take a string, it could take any type and use this for a “scope”. It intrigued me, so I got playing.

The Basics Of Scopes In ILogger

I’m not really sure on the use of “scope” as Microsoft has used it here. Scope when talking about logging seems to imply either the logging level, or more likely which classes and use a particular logger. But scope as Microsoft has defined it in ILogger is actually to do with adding extra messaging onto a log entry. It’s somewhat metadata-ish, but it tends to lend itself more like a stacktrace from within a single method.

To turn on scopes, first you need to actually be using a logger that implements them – not all do. And secondly you need to know if that particular logger has a setting to then turn them on if they are not already on by default. I know popular loggers like NLog and Serilog do use scopes, but for now we are just going to use the Console logger provided by Microsoft. To turn it on, when configuring our logger we just use the “IncludeScopes” flag.

First I’ll give a bit of code to look at as it’s probably easier than me rambling on trying to explain.

Now when we run this, we end up with a log like so :

So for this particular logger, what it does is it appends it before the logs. But it’s important to note that the scope messages are stored separately, it just so happens that this loggers only way of reporting is to push it all into text. You can see the actual code for the Console logger by Microsoft here : https://github.com/aspnet/Logging/blob/dev/src/Microsoft.Extensions.Logging.Console/ConsoleLogScope.cs. So we can see it basically appends it in a hierarchical fashion and doesn’t actually turn it into a string message until we actually log a message.

If for example you were using a Logger that wrote to a database, you could have a separate column for scope data rather than just appending it to the log message. Same thing if you were using a logging provider that allowed some sort of metadata field etc.

Note :  I know that the scope message is doubled up. This is because of the way the ConsoleLogger holds these messages. It actually holds them as a key value pair of string and object. The way it gets the key is by calling “ToString()” on the message… which basically calls ToString() on a string. It then writes out the Key followed by the Value, hence the doubling up. 

So It’s Basically Additional Metadata?

Yes and no. The most important thing about scopes is it’s hierarchy. Every library I’ve looked at that implements scopes implements them in a way that there is a very clear hierarchy to the messages. Plenty of libraries allow you to add additional information with an exception/error message, but through scopes we can determine where the general area of our code got up to and the data around it without having to define private variables to hold this information.

Using Non String Values

The interesting thing is that BeginScope actually accepts any object as a scope, but it’s up to the actual Logger to decide how these logs are written. So for example, if I pass a Dictionary of values to a BeginScope statement, some libraries may just ToString() anything that’s given to them, others may check the type and decide that it’s better represented in a different format.

If we take a look at something like NLog. We can see how they handle scopes here : https://github.com/NLog/NLog.Extensions.Logging/blob/master/src/NLog.Extensions.Logging/Logging/NLogLogger.cs#L269. Or if I take the code and paste it up :

We can see it checks if the state message is a type of IEnumerable<KeyValuePair<string, object>> and then throws it off to another method (Which basically splits open the list and creates a nicer looking message). If it’s anything else (Likely a string), we just push the message as is. There isn’t really any defined way provided by Microsoft on how different types should be handled, it’s purely up to the library author.

How About XYZ Logging Library

Because logging scopes are pretty loosely defined within the ASP.net Core framework, it pays to always test how scopes are handled within the particular logging library you are using. There is no way for certain to say that scopes are even supported, or that they will be output in a way that makes any sense. It’s also important to look at the library code or documentation, and see if there is any special handling for scope types that you can take advantage of.

There is a current proposal that’s getting traction (By some) to make it into C# 8. That’s “default interface methods”. Because at the time of writing, C# 8 is not out, nor has this new language feature even “definitely” made it into that release, this will be more about the proposal and my two cents on it rather than any specific tutorial on using them.

What Are Default Interface Methods?

Before I start, it’s probably worth reading the summary of the proposal on Github here : https://github.com/dotnet/csharplang/issues/288. It’s important to remember is that this is just a proposal so not everything described will actually be implemented (Or implemented in the way it’s initially being described). Take everything with a grain of salt.

The general idea is that an interface can provide a method body. So :

A class that implements this interface does not have to implement methods where a body has been provided. So for example :

The interesting thing is that the class itself does not have the ability to run methods that have been defined and implemented on an interface. So :

The general idea is that this will now allow you to do some sort of multiple inheritance of behaviour/methods (Which was previously unavailable in C#).

There are a few other things that then are required (Or need to become viable) when opening this up. For example allowing private level methods with a body inside an interface (To share code between default implementations).

Abstract Classes vs Default Interface Methods

While it does start to blur the lines a bit, there are still some pretty solid differences. The best quote I heard about it was :

Interfaces define behaviour, classes define state.

And that does make some sense. Interfaces still can’t define a constructor, so if you want to share constructor logic, you will need to use an abstract/base class. An interface also cannot define class level variables/fields.

Classes also have the ability to define accessibility of it’s members (For example making a method protected), whereas with an interface everything is public. Although part of the proposal is extending interfaces with things like the static keyword and protected, internal etc (I really don’t agree with this).

Because the methods themselves are only available when you cast back to the interface, I can’t really see it being a drop in replacement for abstract classes (yet), but it does blur the lines just enough to ask the question.

My Two Cents

This feels like one of those things that just “feels” wrong. And that’s always a hard place to start because it’s never going to be black and white. I feel like interfaces are a very “simple” concept in C# and this complicates things in ways which I don’t see a huge benefit. It reminds me a bit of the proposal of “Primary Constructors” that was going to make it into C# 6 (See more here : http://www.alteridem.net/2014/09/08/c-6-0-primary-constructors/). Thankfully that got dumped but it was bolting on a feature that I’m not sure anyone was really clamoring for.

But then again, there are some merits to the conversation. One “problem” with interfaces is that you have to implement every single member. This can at times lock down any ability to extend the interface because you will immediately blow up any class that has already inherited that interface (Or it means your class becomes littered with throw new NotImplementedException() ).

There’s even times when you implement an interface for the first time, and you have to pump out 20 or so method implementations that are almost boilerplate in content. A good example given in the proposal is that of IEnumerable. Each time you implement this you are required to get RSI on your fingers by implementing every detail. Where if there was default implementations, there might be only a couple of methods that you truly want to make your own, and the default implementations do the job just fine for everything else.

All in all, I feel like the proposal should be broken down and simplified. It’s almost like an American political bill in that there seems to be a lot in there that’s causing a bit of noise (allowing static, protected members on interfaces etc). It needs to be simplified down and just keep the conversation on method bodies in interfaces and see how it works out because it’s probably a conversation worth having.

What do you think? Drop a comment below.

While helping a new developer get started with ASP.net Core, they ran into an interesting exception :

InvalidOperationException: Cannot consume scoped service from singleton.

I found it interesting because it’s actually the Service DI of ASP.net Core trying to make sure you don’t trip yourself up. Although it’s not foolproof (They still give you enough rope to hang yourself), it’s actually trying to stop you making a classic DI scope mistake. I thought I would try and build up an example with code to first show them, then show you here! Of course, if you don’t care about any of that, there is a nice TL;DR right at the end of this post!

The Setup

So there is a little bit of code setup before we start explaining everything. The first thing we need is a “Child” service :

The reason we have a property here called “CreationCount” is because later on we are going to test if a service is being created on a page refresh, or it’s re-using existing instances (Don’t worry, it’ll make sense!)

Next we are going to create *two* parent classes. I went with a Mother and Father class, for no other reason that the names seemed to fit (They are both parent classes). They also will have a creation count, and reference the child service.

Finally, we should create a controller that simply outputs how many times the services have been created. This will help us later in understanding how our service collection is created and, in some circumstances, shared.

The Scoped Service Problem

The first thing we want to do, is add a few lines to the ConfigureServices method of our startup.cs. This first time around, all services will be singletons.

Now, let’s load our page and refresh if a few times and see what the output is. The results of 3 page refreshes look like so :

So this makes sense. A singleton is one instance per application, so no matter how many times we fresh the page, it’s never going to create a new instance.

Let’s change things up, let’s make everything transient.

And then we load it up 3 times.

It’s what we would expect. The “parent” classes go up by 1 each time. The child actually goes up by 2 because it’s being “requested” twice per page load. One for the “mother” and one for the “father”.

Let’s instead make everything scoped. Scoped means that a new instance will be created essentially per page load (Atleast that’s the “scope” within ASP.net Core).

And now let’s refresh out page 3 times.

OK this makes sense. Each page load, it creates a new instance for each. But unlike the transient example, our child creation only gets created once per page load even though two different services require it as a dependency.

Right, so we have everything as scoped. Let’s try something. Let’s say that we decide that our parents layer should be singleton. There could be a few reasons we decide to do this. There could be some sort of “cache” that we want to use. It might have to hold state etc. The one reason we don’t want to make it singleton is for performance. Very very rarely do I actually see any benefit from changing services to singletons. And the eventual screw ups because you are making things un-intuitively singletons is probably worse in the long run.

We change our ConfigureServices method to :

We hit run and….

So because our ChildService is scoped, but the FatherService is singleton, it’s not going to allow us to run. So why?

If we think back to how our creation counts worked. When we have a scoped instance, each time we load the page, a new instance of our ChildService is created and inserted in the parent service. Whereas when we do a singleton, it keeps the exact same instance (Including the same child services). When we make the parent service a singleton, that means that the child service is unable to be created per page load. ASP.net Core is essentially stopping us from falling in this trap of thinking that a child service would be created per page request, when in reality if the parent is a singleton it’s unable to be done. This is why the exception is thrown.

What’s the fix? Well either the parent needs to be scoped or transient scope. Or the child needs to be a singleton itself. The other option is to use a service locator pattern but this is not really recommended. Typically when you make a class/interface singleton, it’s done for a very good reason. If you don’t have a reason, make it transient.

The Singleton Transient Trap

The interesting thing about ASP.net Core catching you from making a mistake when a scoped instance is within a singleton, is that the same “problem” will arise from a transient within a singleton too.

So to refresh your memory. When we have all our services as transient :

We load the page 3 times, and we see that we create a new instance every single time the class is requested.

So let’s try something else. Let’s make the parent classes singleton, but leave the child as a transient.

So we load the page 3 times. And no exception but…

This is pretty close as having the child as scoped with the parents singletons. It’s still going to give us some unintended behaviour because the transient child is not going to be created “everytime” as we might first think. Sure, it will be created everytime it’s “requested”, but that will only be twice (Once for each of the parents).

I think the logic behind this not throwing an exception, but scoped will, is that transient is “everytime this service is requested, create a new instance”, so technically this is correct behaviour (Even though it’s likely to cause issues). Whereas a “scoped” instance in ASP.net Core is “a new instance per page request” which cannot be fulfilled when the parent is singleton.

Still, you should avoid this situation as much as possible. It’s highly unlikely that having a parent as a singleton and the child as transient is the behaviour you are looking for.

TL;DR;

A Singleton cannot reference a Scoped instance.

 

Astute observers will note that .NET Core has been out for quite some time now, with no “GUI” option of building applications. This seems rather deliberate, with Microsoft instead intending to focus on web applications instead of desktop apps. This is also coupled with the fact that to create a cross platform GUI library is easier said then done. Creating something that looks good on Windows, Mac and Linux is no easy feat.

And that’s where Eto.Forms jumps in. It’s a GUI library that uses the native toolkit of each OS/Platform through a simple C# API. Depending on where it is run, it will use WPF, MonoMac or GTK#. It isn’t necessarily using .NET Core (Infact it definitely isn’t, it uses Mono on non Windows platforms), but because one of the main reasons to switch to Core is it’s ability to be cross platform and work on non-windows OS, I thought I would have a little play with it.

In this first post, we will just be working out how to do a simple test application. In future posts, we will work out how to run platform specific code and how better to deploy our application that doesn’t involve a tonne of dependencies to be installed on the machine (For example bundling mono with your application). I’ve spent quite a bit tinkering with this and so far, definitely so good.

Setup Within Visual Studio

When using Visual Studio, we want to take advantage of code templates so that our “base” solution is created for us.

Inside Visual Studio, go to Tools -> Extensions and Updates. Hit “Online” on the left hand side, then type in “Eto.Forms” in your search box.

Go ahead and install the Eto.Forms Visual Studio Addin. It’s highly likely after install you will also need to restart Visual Studio to complete the installation.

When you are back in Visual Studio, go ahead and create a new project. You should now have a new option to create an “ETO Forms Project”.

After selecting this, you are going to be given a few options on the project. The first is that do you want all 3 platforms to be within a single application, or a separate application for each. For the purpose of this demo we are going to go ahead and do 3 separate projects. The only reason we do this is so that building becomes easier to “see” because you have a different bin folder for each platform. As a single project, everything gets built into the same bin folder so it just looks a bit overwhelming.

Next you are given options on how you want to define your layout. And here’s where you might tune out. If you are expecting some sort of WYSIWYG designer, it ain’t happening. You can define everything with C# code (Sort of like how Winforms would generate a tonne of code in the background for you), you can use XAML, or you can use JSON. Me personally, I’m a WinForm kinda guy so I prefer defining the forms in C# code. So I’m going to go ahead and use that in this example. I can definitely see in the future someone building a tool ontop that simply generates the underlying code, but for now, you’ll have to live with designing by hand.

Now we should be left with 4 projects. The first being our “base” project. This defines the forms and shared code no matter which platform we actually end up running on. The other 3 projects are for Linux, Mac and Windows respectively. As I said earlier, if we select the option to have a “single” project, we actually end up with two projects. One that contains shared code, and one that actually launches the thing (But builds for all 3 platforms).

In each of these .csproj files, you will find a line like :

Which basically tells it which platform to build for. And again, you can combine them within the same project to build for two different platforms within the single project.

Something that I was super surprised about is that there is no mac requirement to do a full build of the mac application. I wasn’t really up to date with how apps within MacOS are typically built, but I thought if they were anything like iOS you would need a mac just to deploy the thing. No such requirement here (Although I suspect you will have  one anyway if you actually intend to develop mac applications).

One big thing. When I actually did all of this, there was a whole host of issues actually getting the project to run. Weirdly enough, updating Visual Studio itself was enough to get everything up and running (You need to do this *before* you create the project). So ensure you are on the latest version of VS. 

So how does it actually look when run? Let’s put Windows and Linux side by side.

Definitely some interesting things right off the bat. Mostly that on Linux, we don’t really use Menu Bars, instead they are similar to a Mac where along the top of the window you have your “File” option etc. Also that the sizing is vastly different between the two, both in height and width! The main point is, we managed to build a GUI application sharing the exact same code, and have it run on Windows and Linux! Cool!

One final thing, you can use Visual Studio to add new forms/components instead of typing them up manually. In reality, it doesn’t really do that much boiler plate for you, but it’s there if you need it. Just right click your shared code project, and select Add New Item as you typically would. You should have a new option of adding an “Eto.Forms View”. From here you are given options on what you are trying to create (A panel, dialog or form), and how you want it to be designed (Code, XAML, Json etc).

Setup From A Terminal/Command Prompt

Alright, so you are one of “those” people that want to use a terminal/command prompt everywhere. Well that’s ok, Eto has you covered! In reality because there is no WYSIWYG designer, there isn’t any great need for Visual Studio if you prefer to use VS Code. Realistically the only difference is how you create the initial project template (Since Visual Studio extensions are going to be off the cards).

The first thing we are going to do is install the Eto.Forms templates. To do that, we run a specific dotnet command in our terminal/command prompt.

To see all available options to us, we can run :

But we are pros and know exactly what we want. So we run :

What this does is create a solution in the current directory (Named after the folder name). It creates a solution for us because of the -sln flag. The -m flag tells us which “mode” we want to use for designing our forms (In our case we want to use code). And finally -s says to create separate projects for each platform. And that’s it! We have a fully templated out project that is essentially identical to the one we created in Visual Studio.

One more thing, you should also be able to create “files” in a similar way to creating them in Visual Studio. So it does a bit of boiler plate legwork for you. To see all available options to you, you can run :

Again, it’s basically creating a class with the right inheritance and that’s about it. But it can be handy.

 

Ever since I posted a quick guide to sending email via Mailkit in .NET Core 2, I have been inundated with comments and emails asking about specific exception messages that either Mailkit or the underlying .NET Core. Rather than replying to each individual email, I would try and collate all the errors with sending emails here. It won’t be pretty to read, but hopefully if you’ve landed here from a search engine, this post will be able to resolve your issue.

I should also note that the solution to our “errors” is usually going to be a change in the way we use the library MailKit. If you are using your own mail library (Or worse, trying to open the TCP ports yourself and manage the connections), you will need to sort of “translate” the fix to be applicable within your code.

Let’s go!

Default Ports

Before we go any further, it’s worth noting the “default” ports that SMTP is likely to use, and the settings that go along with them.

Port 25
25 is typically used a clear text SMTP port. Usually when you have SMTP running on port 25, there is no SSL and no TLS. So set your email client accordingly.

Port 465
465 is usually used for SMTP over SSL, so you will need to have settings that reflect that. However it does not use TLS.

Port 587
587 is usually used for SMTP when using TLS. TLS is not the same as SSL. Typically anything where you need to set SSL to be on or off should be set to off. This includes any setting that goes along the lines of “SSLOnConnect”. It doesn’t mean TLS is less secure, it’s just a different way of creating the connection. Instead of SSL being used straight away, the secure connection is “negotiated” after connection.

Settings that are relevant to port 587 usually go along the lines of “StartTLS” or simply “TLS”. Something like Mailkit usually handles TLS for you so you shouldn’t need to do anything extra.

Incorrect SMTP Host

System.Net.Sockets.SocketException: No such host is known

This one is an easy fix. It means that you have used an incorrect SMTP hostname (Usually just a typo). So for example using smp.gmail.com instead of smtp.gmail.com. It should be noted that it isn’t indicative that the port is wrong, only the host. See the next error for port issues!

Incorrect SMTP Port

System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: A socket operation was attempted to an unreachable network [IpAddress]:[Port]

Another rather simple one to fix. This message almost always indicates you are connecting to a port that isn’t open. It’s important to distinguish that it’s not that you are connecting with SSL to a non SSL port or anything along those lines, it’s just an out and out completely wrong port.

It’s important that when you log the exception message of this error, you check what port it actually says in the log. I can’t tell you how many times I “thought” I was using a particular port, but as it turned out, somewhere in the code it was hardcoded to use a different port.

Another thing to note is that this error will manifest itsef as a “timeout”. So if you try and send an email with an incorrect port, it might take about 30 seconds for the exception to actually surface. This is important to note if your system slows to a crawl as it gives you a hint of what the issue could be even before checking logs.

Forcing Non-SSL On An SSL Port

MailKit.Net.Smtp.SmtpProtocolException: The SMTP server has unexpectedly disconnected.

I want to point out this is a MailKit exception, not one from .NET. In my experience this exception usually pops up when you set Mailkit to not use SSL, but the port itself requires SSL.

As an example, when connecting to Gmail, the port 465 is used for SSL connections. If I try and connect to Gmail with the following Mailkit code :

That last parameter that’s set to false is telling Mailkit to not use SSL. And what do you know, we get the above exception. The easy fix is obviously to change this to “true”.

Alternatively, you might be getting confused about TLS vs SSL. If the port says to connect using SSL, then you should not “force” any TLS connection.

Connecting Using SSL On A TLS Port (Sometimes)

MailKit.Security.SslHandshakeException: An error occurred while attempting to establish an SSL or TLS connection.

Another Mailkit specific exception. This one can be a bit confusing, especially because the full error log from MailKit talks about certificate issues (Which it totally could be!), but typically when I see people getting this one, it’s because they are trying to use SSL when the port they are connecting to is for TLS. SSL != TLS.

So instead of trying to connect using MailKit by forcing SSL, just connect without passing in an SSL parameter.

I should note that you will also get this exception in Mailkit if you try and force the SSLOnConnect option on a TLS Port. So for example, the following will not work :

Again, sometimes this is because the SSL connection truly is bogus (Either because the certificate is bad, expired etc), or because you are connecting to a port with no security whatsoever. But because of the rise of TLS, I’m going to say the most likely scenario is that you are trying to force SSL on a TLS port.

Another error that you can sometimes get with a similar setup is :

Handshake failed due to an unexpected packet format

This is an exception that has a whole range of causes, but the most common is forcing an SSL connection on a TLS port. If your SMTP port supports “TLS”, then do not set SSL to true.

Forcing No TLS On A TLS Port

System.NotSupportedException: The SMTP server does not support authentication.

This one is a little weird because you basically have to go right out of your way to get this exception. The only way I have seen this exception come up when you are telling Mailkit to go out of it’s way to not respect any TLS messages it gets and try and ram authentication straight away.

So for example, the following code would cause an issue :

Incorrect Username And/Or Password

MailKit.Security.AuthenticationException: AuthenticationInvalidCredentials: 5.7.8 Username and Password not accepted.

Goes without saying, you have the wrong username and password. This is not going to be any connection issues. You definitely have the right host and port, and the right SSL/TLS settings, but your username/password combination is wrong.

Other Errors?

Have something else going haywire? Drop a comment below!

Many years back, I actually started programming so that I could cheat at an online web browser based game (I know, I know). I started in Pascal/Delphi, but quickly moved onto C#. Back then, I was using things like HttpWebRequest/Response to load the webpage, and using raw regex to parse it. Looking back, some of my earlier apps I wrote were just a complete mess. Things have changed, and it’s now easier than ever to automate web requests using C#/.NET Core. Let’s get started.

Loading The Webpage

Let’s start by creating a simple console application. Immediately, let’s go ahead and change this to an async entry point. So from this :

We just make it async and return Task :

However if you’re using an older version of C#, you will get an error message that there is no valid entry point. Something like :

In which case you need to do something a little more complex. You need to turn your main method to look like the following :

What this essentially does is allow you to use async methods in a “sync” application. I’ll be using a proper async entry point, but if you want to go along with this second method, then you need to put the rest of this tutorial into your “MainAsync”.

For this guide I’m actually going to be loading my local news site, and pulling out the “Headline” article title and showing it on a console screen. The site itself is http://www.nzherald.co.nz/.

Loading the webpage is actually rather easy. I jam the following into my main method :

And run it to check the results :

Great! So we loaded the page via code, and we just spurted it all out into the window. Not that interesting so far, we probably need a way to extract some content from the page.

Extracting HTML Content

As I said earlier, in the early days I just used raw regex and with a bit of trial and error got things working. But there are much much easier ways to read content these days. One of these is to use HtmlAgilityPack (HAP). HAP has been around for years, which means that while it’s battle tested and is pretty bug free, it’s usage can be a bit “archaic”. Let’s give it a go.

To use HAP, we first need to install the nuget package. Run the following from your Package Manager Console :

The first thing we need to do is feed our HTML into an HtmlDocument. We do that like so :

Now given I wanted to try and read the title of the “headline” article on NZHerald, how do we go about finding that piece of text?

HAP (And most other parsers) work using a thing called “XPath“. It’s this sort of query language that tells us how we should traverse through an XML document. Because HTML is pretty close to XML, it’s also pretty handy here. Infact, a big part of what HAP does is try and clean up the HTML so it’s able to be parsed more or less as an HTML document.

Now the thing with XPath within an HTML document is that it can be a bit trial and error. Your starting point should be pulling up your browsers dev tools, and just inspecting the divs/headers/containers around the data you want to extract.

In my case, the first thing I notice is that the headline article is contained within a div that has a class of pb-f-homepage-hero . As I scroll down the page, this class is actually used on a few different other containers, but the first one in the HTML document is what I wanted. Ontop of that, our text is then inside a nice h3  tag within this container.

So given these facts, (And that I know a bit of XPath off the top of my head), the XPath we are going to use is :

Let’s break it down a bit.

//  says that we can start anywhere within the document (Not just the first line).
div  says that we are looking for a div that…
[contains(@class,'pb-f-homepage-hero')]  contains a “class” attribute that has the text ‘pb-f-homepage-hero’ somewhere within it.
//h3  says that within this div element, somewhere in there (Doesn’t have to be the first child, could be deeper), there will be an h3
()[1]  says just get me the first one.

We can actually test this by going here : http://videlibri.sourceforge.net/cgi-bin/xidelcgi. We can paste our HTML into the first window, our xpath into the second, and the results of our expression will come up below. There are a few XPath testers out there, but I like this one in particular because it allows for some pretty busted up HTML (Others try and validate that it’s valid XML).

So we have our XPath, how about we actually use it?

So really we can paste our entire XPath into a “SelectSingleNode” statement, and we use the “InnerText” property to get just the textual content within that node (Rather than the HTML).

If we let this run :

Done! We have successfully grabbed the headline news title! (Which is currently that a political party within NZ has nominated Simon Bridges as it’s new leader).