It feels like not long ago we were talking about all the goodies in .NET 5, and here we are already jumping into .NET 6. It actually made me go back and look at the .NET Framework versions on Wikipedia here. Admittedly back then, minor versions of the framework sometimes contained huge changes. For example async/await was added in version 4.5, which these days would suggest a “minor” update, but obviously was pretty huge. But even still, version 1.0 to version 4.8 was 17 years in the making.

The first version of .NET Core was released in 2016, and here we are in 2021, just 5 years later, and we are already up to seeing preview versions of .NET Core 6. It really speaks to not only Microsoft’s commit to move fast, but I think just the overall cadence of modern software development. Gone are the days of developers sitting in cubicles and sitting on work for three years until a a major release.

You can grab .NET 6 Preview 1 here : https://dotnet.microsoft.com/download/dotnet/6.0

As for what’s new. Well generally the first preview release of a new .NET Version is typically setting the stage for what’s to come and doesn’t necessarily contain a lot of “toys” to play with. With that being said, some of the features that did make it in were :

  • The first iteration of moving Xamarin into .NET to unify the platforms. e.g. Being able to build Android/IOS applications in .NET.
  • A first crack at using Blazor for desktop applications (From early reading, this seems very close to how you might use Electron, e.g. It’s still a web control view on desktop).
  • There seems to be talk about better hot reload functionality. I can’t find that much information on this. The .NET CLI already has “dotnet watch“, but this is more of a complete build rather than a nice iterative hot reload.
  • Improvement to Single File Apps so that they actually execute from that single file rather than extracting into temp directories. This was already the case for single file applications for Linux in .NET 5, but in .NET 6, this functionality has been extended for Windows and Mac.
  • There is no appsettings.json auto complete for commonly used configuration such as logging, host filtering, kestrel setup etc.
  • WPF is now supported on ARM64.

The full release as always is here : https://devblogs.microsoft.com/dotnet/announcing-net-6-preview-1/

Overall, probably not a heck of a lot if you are a web developer. One of the major themes of .NET 6 (And even .NET 5), is unifying the various platforms and frameworks that sit under the .NET banner. In .NET 5, it was mostly desktop development, and in .NET 6, it’s a lot of mobile development with Xamarin. It doesn’t mean there won’t be something in future preview versions of course, but for now, get excited about mobile dev!

ENJOY THIS POST?
Join over 3.000 subscribers who are receiving our weekly post digest, a roundup of this weeks blog posts.
We hate spam. Your email address will not be sold or shared with anyone else.


This post is part of a series on using Auth0 with an ASP.NET Core API, it’s highly recommended you start at part 1, even if you are only looking for something very specific (e.g. you came here from Google). Skipping parts will often lead to frustration as Auth0 is very particular about which settings and configuration pieces you need.

Part 1 – Auth0 Setup
Part 2 – ASP.NET Core Authentication
Part 3 – Swagger Setup


It’s very rare to build an API in .NET Core, and not use Swagger. After all, it’s the easiest self documenting tool available to developers, and provides a great way to test API’s without using a third party tool such as Postman. Setting up Swagger for general use is not really part of this article series, but we already have a previous article on the subject here : https://dotnetcoretutorials.com/2020/01/31/using-swagger-in-net-core-3/. If you are new to using Swagger, have a read as this piece of the Auth0 article series will cover setting up Swagger to work with Auth0, but not setting up Swagger itself!

With that out of the way, let’s jump right in.

Adding Auth0 Config To Swagger

In our startup.cs file, and inside the ConfigureServices method, we will have something similar to “AddSwaggerGen”. What we need to do is add a SecurityDefinition to Swagger. What this does is define how our API is authenticated, and how Swagger can authorize itself to make API calls. At a high level, it’s telling Swagger that “Hey, you need a token to call this API, here’s how to get one”.

The full code looks like so :

services.AddSwaggerGen(c =>
{
    c.SwaggerDoc("v1",
            new OpenApiInfo
            {
                Title = "API",
                Version = "v1",
                Description = "A REST API",
                TermsOfService = new Uri("https://lmgtfy.com/?q=i+like+pie")
            });

    c.AddSecurityDefinition("Bearer", new OpenApiSecurityScheme
    {
        Name = "Authorization",
        In = ParameterLocation.Header,
        Type = SecuritySchemeType.OAuth2,
        Flows = new OpenApiOAuthFlows
        {
            Implicit = new OpenApiOAuthFlow
            {
                Scopes = new Dictionary<string, string>
                {
                    { "openid", "Open Id" }
                },
                AuthorizationUrl = new Uri(Configuration["Authentication:Domain"] + "authorize?audience=" + Configuration["Authentication:Audience"])
            }
        }
    });
});

What we are really adding is that SecurityDefinition. It’s somewhat beyond the scope of this article to really get into the nitty gritty of what each of these properties do, but this is the correct setup for Auth0. Also notice that our AuthorizationUrl is using our previous configuration that we set up to get .NET Core Authentication working.

Now move to the Configure method of your startup.cs. You need to modify your UseSwaggerUI call to look like so :

app.UseSwaggerUI(c =>
{
    c.SwaggerEndpoint("/swagger/v1/swagger.json", "API");
    c.OAuthClientId(Configuration["Authentication:ClientId"]);
});

Again, this is using a configuration variable that we set up earlier. All going well, if you open Swagger now, you should see a button saying Authorize at the top like so :

Clicking this and authenticating will redirect you back to Swagger, upon which you can make API calls that will send your bearer token.

If you get the following error :

Callback URL mismatch. The provided redirect_uri is not in the list of allowed callback URLs

It’s because you need to add your swagger URL (e.x. https://localhost:5001/swagger/oauth2-redirect.html) to the list of Allowed Callback URLs for your Auth0 application.

Now here’s where things diverge. If you are using the Authorize attribute on controllers (e.g. You have [Authorize] on top of every Controller class), then you are good to go. You should be able to tell because for each controller action inside Swagger, there will be a padlock icon indicating that authentication is required.

If you don’t see this padlock icon, it means that either you don’t have the correct Authorize attribute applied *or* you are using my method of applying Authorize globally. If it’s the former, then apply the Authorize attribute. If it’s the latter, continue reading below!

Adding SecurityRequirementsOperationFilter To Swagger

Swagger identifies which methods require authentication by looking for the [Authorize] attribute on controllers. But of course, if you are applying this globally as a convention like we mentioned earlier, this attribute won’t be there. So instead, we have to give Swagger a hand.

Add a class in your API project called “SecurityRequirementsOperationFilter”, and paste the following :

public class SecurityRequirementsOperationFilter : IOperationFilter
{
    /// <summary>
    /// Applies the this filter on swagger documentation generation.
    /// </summary>
    /// <param name="operation"></param>
    /// <param name="context"></param>
    public void Apply(OpenApiOperation operation, OperationFilterContext context)
    {
        // then check if there is a method-level 'AllowAnonymous', as this overrides any controller-level 'Authorize'
        var anonControllerScope = context
                .MethodInfo
                .DeclaringType
                .GetCustomAttributes(true)
                .OfType<AllowAnonymousAttribute>();

        var anonMethodScope = context
                .MethodInfo
                .GetCustomAttributes(true)
                .OfType<AllowAnonymousAttribute>();

        // only add authorization specification information if there is at least one 'Authorize' in the chain and NO method-level 'AllowAnonymous'
        if (!anonMethodScope.Any() && !anonControllerScope.Any())
        {
            // add generic message if the controller methods dont already specify the response type
            if (!operation.Responses.ContainsKey("401"))
                operation.Responses.Add("401", new OpenApiResponse { Description = "If Authorization header not present, has no value or no valid jwt bearer token" });

            if (!operation.Responses.ContainsKey("403"))
                operation.Responses.Add("403", new OpenApiResponse { Description = "If user not authorized to perform requested action" });

            var jwtAuthScheme = new OpenApiSecurityScheme
            {
                Reference = new OpenApiReference { Type = ReferenceType.SecurityScheme, Id = "Bearer" }
            };

            operation.Security = new List<OpenApiSecurityRequirement>
            {
                new OpenApiSecurityRequirement
                {
                    [ jwtAuthScheme ] = new List<string>()
                }
            };
        }
    }
}

This looks a bit over the top, but actually it’s just telling Swagger that unless it sees an “AllowAnonymous” attribute on an action or a controller, that we can assume it’s supposed to be authenticated. It’s essentially flipping things on it’s head and saying everything requires authentication unless I say so.

Now back in our ConfigureServices method of our startup.cs, we can go :

services.AddSwaggerGen(c => 
{
    //All the other stuff. 
    c.OperationFilter<SecurityRequirementsOperationFilter>();
});

Which will of course add in our new filter to our swagger docs. This means that now, when we use Swagger, by default, all actions will require a JWT token. Perfect!

ENJOY THIS POST?
Join over 3.000 subscribers who are receiving our weekly post digest, a roundup of this weeks blog posts.
We hate spam. Your email address will not be sold or shared with anyone else.


This post is part of a series on using Auth0 with an ASP.NET Core API, it’s highly recommended you start at part 1, even if you are only looking for something very specific (e.g. you came here from Google). Skipping parts will often lead to frustration as Auth0 is very particular about which settings and configuration pieces you need.

Part 1 – Auth0 Setup
Part 2 – ASP.NET Core Authentication
Part 3 – Swagger Setup


Now that we have our Auth0 tenant all set up, it’s time to actually start authenticating users on our API, and validating their JWT tokens. Let’s go!

Setting Up Auth0 With ASP.NET Core Authentication

The first thing we need to do is install the Microsoft Nuget package that validates JWT tokens for us. So from our Package Manager Console we can run :

Install-Package Microsoft.AspNetCore.Authentication.JwtBearer

Next, head to our startup.cs file, and inside our ConfigureServices method, we will add the following :

services.AddAuthentication(options =>
{
    options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme;
    options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme;
}).AddJwtBearer(options =>
{
    options.Authority = Configuration["Authentication:Domain"];
    options.Audience = Configuration["Authentication:Audience"];
});

This sets up our JWT authentication to be validated against Auth0. When I did all of this for the first time I thought “I must be missing something here…”. But really that’s it. Any JWT token is validated against Auth0 using the configuration we set up earlier. Too easy!

Next, in our Configure method, we need two additional calls in our pipeline :

app.UseAuthentication();
app.UseAuthorization();

Ordering is important! The call to Authentication must happen before the call to Authorization. Authentication is the act of “authenticating” who someone is, and essentially storing a validated identity against that request. Authorization is the act of authorizing a user against a resource. If you have not authenticated (e.g. Logged in), then how can you be authorized?

The overall order within this method is important too. You should obviously authenticate before you make a call to a controller etc.

Adding Authorize Attribute

To require your controllers to have a logged in user, we must go and place the “Authorize” attribute on each controller like so :

[Authorize]
public class ContactController : ControllerBase
{
}

However, there are a couple of problems with this :

  • You now have to go back and back-add it to all controllers.
  • What if a new controller is added, and someone forgets to add this attribute.

That last point is I think the most important. What we want to do is reverse the Authorize attribute to be opt-out, not opt-in. By default, everything should be locked down to logged in users. Luckily there is a way for us to do just that.

In your startup.cs, inside your ConfigureServices method, you should have a call to “AddControllers” or similar like so :

services.AddControllers()

However, you can also use this call to add in filters that are applied globally, without you having to add the attribute manually to each controller. To do that with our Authorize attribute, we do the following :

services.AddControllers(options =>
{
    var policy = new AuthorizationPolicyBuilder()
        .RequireAuthenticatedUser()
        .Build();

    options.Filters.Add(new AuthorizeFilter(policy));
})

Now the AuthorizeFilter is added globally for every controller within our solution!

Of course the next question will be, what if you want a controller to opt out? We can just use the AllowAnonymous attribute like so :

[AllowAnonymous]
public class AnonController : ControllerBase
{
}

Testing ASP.NET Core Authentication

At this point, your API is actually all set up to authenticate against JWT tokens. In the next step, we are going to talk about how to wire up Swagger to allow you to generate valid test tokens within the Swagger interface. But if you can’t wait that long, or you don’t use Swagger, then you can actually generate test tokens right from Auth0 itself.

Inside the Auth0 Dashboard, select “APIs” from the left hand menu, open the settings for your API and go to the “Test” tab. There, the second box actually contains a valid JWT token that you can use for testing. It’s generated each time you load this page, so it’s good to go immediately. Feel free to test your API at this point with the JWT token here, and validate that everything is set up correctly.

Next Steps

Theoretically, our API is now secured using Auth0. But in 99% of my projects, I use Swagger to test against my API. For that, I want to be able to generate a valid Auth0 JWT token to use for testing, without having to log into Auth0 or use Fiddler on my front end application to intercept a valid token. The next part in our series will investigate doing exactly that : https://dotnetcoretutorials.com/2021/02/14/using-auth0-with-an-asp-net-core-api-part-3-swagger/

ENJOY THIS POST?
Join over 3.000 subscribers who are receiving our weekly post digest, a roundup of this weeks blog posts.
We hate spam. Your email address will not be sold or shared with anyone else.

I’ve recently had to set up a new project using Auth0 as an “Identity As A Service” provider. Essentially, Auth0 provides an authentication service using an OAuth2 flow, meaning I don’t have to store passwords, worry about passwords resets, or implement my own two factor authentication. Everything about authenticating a user is handled by Auth0, it’s great!

What’s not great is their documentation. I’ve had to use Auth0 (And Azure AD B2C) in a tonne of projects over the years. And every time, I’m reminded that their documentation just plain sucks. At a guess, I think it’s because you only do it once. So if you set up Auth0 for your product, you’re only doing that once and you’ll never have to do it again. So any pains in the documentation you quickly get over. Except if you’re me! Because I work across a whole range of projects on a contract basis, I may do a new Auth0 setup up to 3 – 4 times per year. And every time, it’s painful.

In this series, I’m going to show you how to authenticate your API using Auth0, from setting up your Auth0 tenant all the way to setting up Swagger correctly. It will serve as a great guide if it’s your first time using Auth0, and for those more experienced, it will provide a good run sheet every time you have to set up a new tenant.


This post is part of a series on using Auth0 with an ASP.NET Core API, it’s highly recommended you start at part 1, even if you are only looking for something very specific (e.g. you came here from Google). Skipping parts will often lead to frustration as Auth0 is very particular about which settings and configuration pieces you need.

Part 1 – Auth0 Setup
Part 2 – ASP.NET Core Authentication
Part 3 – Swagger Setup


Creating An Auth0 API

The first thing we need to do is create a new “API” within the Auth0 dashboard. From Auth0, click the APIs menu item, click “Create API” and fill it in similar to the following :

The Name field can be anything, and is purely used within the portal. This might be useful if you have multiple different API’s that will authenticate differently, but for the most part, you can probably name it your product.

The “Identifier” is a little more tricky. It plays a similar role to the above in that it identifies which API is being authenticated for, but… Again, if you have one API it’s not too important. I typically do https://myproductname. It does not have to be a URL at all however, but this is just my preference.

Leave the signing algorithm as is and hit Create!

Copy the Identifier you used into a notepad for safe keeping as we will need it later.

Creating Your Auth0 Application

Next we need to set up our Auth0 Application. An application within the context of Auth0 can be thought of as a “solution”. Within your solution you may have multiple API’s that can be authenticated for, but overall, they are all under the same “Application”.

By default, Auth0 has an application created for you when you open an account. You can rename this to be the name of your product like so :

Also take note of your “Domain” and “ClientId”. We will need these later so copy and paste them out into your notepad file.

Further down, make your “Application Type” set to “Single Page Application”.

On this same settings page for your application, scroll down and find the “Allowed Callback URLs”. This should be set up to allow a call back to your front end (e.g. React, Angular etc). But it should also allow for a Swagger callback. (Confusing, I know). But to put it simply, pop in the URL of your local web application *and* the domain of your API application like so :

Remember to hit “Save Changes” right at the bottom of the page.

Adding Configuration To ASP.NET Core

In our .NET Core solution, open up the appsettings.json file. In there, add a JSON node like so :

"Authentication": {
  "Domain": "https://mydomain.us.auth0.com/",
  "Audience": "https://myproduct",
  "ClientId": "6ASJKHjkhsdf776234"
}

We won’t actually use this configuration anywhere except in our startup method, so for now, don’t worry about creating a C# class to represent this configuration.

Next Steps

So far we’ve set up everything we need on the Auth0 side, and we’ve grabbed all the configuration values and put them into ASP.NET Core. Now, we need to set up everything related to authentication inside our .NET Core App. You can check out the next step in the series here : https://dotnetcoretutorials.com/2021/02/14/using-auth0-with-an-asp-net-core-api-part-2-asp-net-core-authentication/

ENJOY THIS POST?
Join over 3.000 subscribers who are receiving our weekly post digest, a roundup of this weeks blog posts.
We hate spam. Your email address will not be sold or shared with anyone else.

Some time back, I wrote a post about PostSharp Threading. I was incredibly impressed by the fact that a complicated task such as thread synchronization had been boiled down to just a couple of C# attributes. While writing the post, I also took a look at the other libraries available from PostSharp, and something that caught my eye was the PostSharp Logging framework. Now I’ve seen my fair share of logging frameworks so at first, I wasn’t that jazzed. Generally speaking when I see a new logging library get released, it’s just another way to store text logs and that’s about it. But PostSharp Logging does something entirely new, without completely re-inventing the wheel.

Of course we are going to dig into all the goodness, but at an overview level. PostSharp Logging is more like a mini APM by automatically logging what’s going on inside your application, rather than just giving you some static “Logger.Error(string message)” method to output logs to. And instead of making you configure yet another logging platform with complicated XML files and boilerplate code, it just hooks into whatever logging framework you are already using. Serilog, Log4Net, and even just plain old ASP.NET Core logger factory are supported with very little setup.

Setting Up Logging

I’ve kind of sold the zero setup time a little bit here so let’s look at actually what’s required.

The first thing we have to do is install the nuget package for our particular logging framework. Now this might get complicated if you are using things like Serilog or Log4Net on top of the .NET Core logger, but for me, I’m just looking to pump all messages to the standard .NET Core output. So all I need to do is install the following two packages :

Install-Package PostSharp.Patterns.Diagnostics
Install-Package PostSharp.Patterns.Diagnostics.Microsoft

Next, I have to do a little bit of work in my program.cs to add the PostSharp logger :

public static void Main(string[] args)
{
    var host = CreateHostBuilder(args).Build();
    var loggerFactory = (ILoggerFactory)host.Services.GetService(typeof(ILoggerFactory));
    LoggingServices.DefaultBackend = new MicrosoftLoggingBackend(loggerFactory);
    host.Run();
}

This might seem a little complicated, but actually you’re just going to be copy and pasting this from the documentation from PostSharp, there actually isn’t much thought involved!

And that’s it! Now we can simply add the [Log] attribute to any method and have it log some pretty juicy stuff. For example, consider the following code :

[Log]
[HttpGet("Hello")]
public async Task Hello([FromQuery]string name)
{
    if(string.IsNullOrEmpty(name))
    {
        return BadRequest("A name is required");
    }

    return Ok($"Hello {name}!");
}

With nothing but the log attribute, I suddenly see these sorts of messages popping up when I call a URL such as /Hello?name=Bob.

dbug: PostSharpLogging.Controllers.TestController[2]
      TestController.Hello("Bob") | Starting.
dbug: PostSharpLogging.Controllers.TestController[4]
      TestController.Hello("Bob") | Succeeded: returnValue = {OkObjectResult}.

Notice how I now capture the method being executed, the parameters being executed, and what the result was. This can be incredibly important because not only are you capturing what methods are running, but you are capturing the input and output of those methods. This could be invaluable if you’re trying to debug under what circumstances a particular method fails or produces an unexpected response.

Writing Detailed APM Style Logging Messages

Earlier I spoke a little bit about how I thought PostSharp.Logging was more like a mini APM rather than a logging framework. That doesn’t mean it can’t log your standard text messages, but at the same time, it has incredible capability to “time” methods and capture exactly what’s going on in your application with very little set up.

All I need to do is create a file in the root of my project called postsharp.config. In it, I add the following :

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.postsharp.org/1.0/configuration">
  <Logging xmlns="clr-namespace:PostSharp.Patterns.Diagnostics;assembly:PostSharp.Patterns.Diagnostics">
    <Profiles>
      <LoggingProfile Name="Detailed" IncludeSourceLineInfo="True" IncludeExecutionTime="True" IncludeAwaitedTask="True">
      </LoggingProfile>
    </Profiles>
  </Logging>
</Project>

It may look confusing at first, but the PostSharp documentation gives you almost all of this out of the box. So what are we now adding to our logs?

  • Capturing the source line info (e.g. What line number is being executed).
  • Capturing the total execution time for a method.
  • Including awaited tasks (More on this later!). But this means that we can actually see when a task is really awaited which is invaluable to solving deadlock issues.

All of this is combined to create named logging profile called “Detailed”. Named profiles are handy because we can now change all of the logging for our project from this one configuration file, instead of going around and modifying Log attributes one by one.

It does mean that we have to modify our Log attribute to look like this :

[Log("Detailed")] // Pass in our log profile name
[HttpGet("Hello")]
public async Task Hello([FromQuery]string name)
{
    if(string.IsNullOrEmpty(name))
    {
        return BadRequest("A name is required");
    }

    return Ok($"Hello {name}!");
}

And now if we run things?

dbug: PostSharpLogging.Controllers.TestController[4]
      TestController.Hello("Bob") | Succeeded: returnValue = {OkObjectResult}, 
      executionTime = 0.40 ms, 
      source = {WeatherForecastController.cs: line 18}.

So now not only are we capturing the input and output, but we are also capturing the total execution time of the method as well as the actual line number of the code. If there was a particular input to this method that caused a slow down or a noticeable performance impact, then we would be able to capture that easily. In fact, let’s test that out now!

Capturing Performance Degradations With PostSharp Logging

I want to create an artificial delay in my application to test how PostSharp Logging identifies this. But before I do this, I want to explain a concept called “Wall Time”.

Wall Time is also sometimes called Wall Clock Time, or even just Real World Time. What it means is that if I’m timing the performance of my application, the only real metric I care about is the actual time a user sits there waiting for a response. So it’s the time from a user say, clicking a button, to actually seeing a response. We call this Wall Time or Wall Clock Time, because if there was a clock on the wall, we could use it to time the response. Now where this can deviate slightly when compared to things such as “CPU Time”. CPU Time refers to how much time the CPU actually spent completing your task. This may differ because the CPU may be juggling work, or it may delay your work because it’s processing someone else’s request, or you may even have an intentional delay in your code.

Confused? Maybe this simplified diagram will help.

Notice how our user in blue sent a request to the CPU, but it was busy servicing our user in red. Once it finished red’s tasks, it then swapped to blue. If you asked the CPU how long it spent working on blue’s task, it will give a very different answer to if you asked the blue user how long they waited. Both timing’s are important, but it’s an important distinction to make when you are building software for end users.

OK, so with that out of the way, why do I bring it up now? Well there is a very large APM product on the market right now that gives timings in CPU Time. While helpful, this was actually incredibly irritating because it doesn’t capture the time a user actually spent waiting. And there is a very easy test for this, and that is to use Task.Delay to simulate the CPU not doing work.

Let’s modify our code to look like so :

[Log("Detailed")]
[HttpGet("Hello")]
public async Task Hello([FromQuery]string name)
{
    if(string.IsNullOrEmpty(name))
    {
        return BadRequest("A name is required");
    }

    if(name == "wade")
    {
        await Task.Delay(1000);
    }

    return Ok($"Hello {name}!");
}

Now if I pass in the name “wade”, I’ll be forced to wait an extra 1000ms before I am given a response. So how does PostSharp log this?

dbug: PostSharpLogging.Controllers.TestController[16]
      TestController.Hello("wade") | Awaiting: asyncCallId = 1, awaitedMethod = Task.Delay
dbug: PostSharpLogging.Controllers.TestController[32]
      TestController.Hello("wade") | Resuming: asyncCallId = 1, awaitedMethod = Task.Delay
dbug: PostSharpLogging.Controllers.TestController[4]
      TestController.Hello("wade") | Succeeded: returnValue = {OkObjectResult}, executionTime = 1038.39 ms

Interesting, the first thing to note is that because I earlier turned on logging for awaited methods, I can now even see when a method is actually awaited, and when it’s resumed. This is really important when working with async/await because not every time you await a method, do you truly await it (But more on that in another post).

Most importantly, look at our execution time! 1038ms. PostSharp is indeed logging the execution time correctly as it pertains to wall time. This is exactly what we want. It may seem like something so simple, but as I’ve said, I know of APM products on the market right now that can’t get this right.

There’s still something more I want to do with this code however. We’re still logging an awful lot when really we just want to capture logging if the performance is degraded. And of course, PostSharp Logging provides us with this. If we modify our logging profile to look like so :

<LoggingProfile Name="Detailed" ExecutionTimeThreshold="200" IncludeSourceLineInfo="True" IncludeExecutionTime="True" IncludeAwaitedTask="True"> 
</LoggingProfile>

We set the ExecutionTimeThreshold to be 200ms. And anything over that we get :

warn: PostSharpLogging.Controllers.TestController[32768]
      TestController.Hello("wade") | Overtime: returnValue = {OkObjectResult}, executionTime = 1012.60 ms, threshold = 200 ms}.

Notice how this is a “Warn” message, not a debug message. Now we can perfectly isolation performance impacts to this particular input, rather than sifting through thousands of logs.

Logging Multiple Methods

Let’s say that you’ve already got a large existing project, but you want to add logging to all controller actions. If we used our code above, we would have to go through copy and pasting our Log attribute everywhere which could be quite the task. And again, if we ever want to remove this logging, we have to go through deleting the attribute.

But PostSharp has us covered with “Multicasting”. Multicasting is the ability to apply the attribute to multiple declarations using a single line of code. And best of all, it allows us to filter where we apply it by using wildcards, regular expressions, or even filtering on some attributes. That means it’s not an all or nothing approach. We can almost fine tune where we log just as well as if we were placing the Log attribute manually on each method.

To get started, create a file called “GlobalLogging.cs” and place it in the root of your project.

Inside, we’re gonna add the following :

using PostSharp.Extensibility;
using PostSharp.Patterns.Diagnostics;

[assembly: Log(AttributePriority = 1, 
    ProfileName = "Detailed",
    AttributeTargetTypes ="MyProjectName.Controllers.*", 
    AttributeTargetMemberAttributes = MulticastAttributes.Public)]

All we are saying is, add the Log attribute, with the ProfileName of “Detailed”, to all target types that are under the controllers namespace. I’m also going to add another filter to say only do this for public methods.

Running my project now, I receive all of the same logging on all of my controller methods, but without having to manually add the Log attribute!

Again, the simplicity of PostSharp stands out. We can add multiple of these global attributes to this file, all with specifically fine tuned wildcards/regexes, and just have it… work. I almost want to write more about all the options you can do with this, but it’s just all so simple and works out of the box, that I’m literally just giving one liners to completely re-invent your logging. It’s really great stuff.

Who Is This Library For?

If you’re working on a software stack that requires you to be constantly managing performance and fine tuning the system, then I think PostSharp Logging is kind of a no brainer. I think the name of “Logging” implies that all it’s really going to do is write text logs for you, but it’s so much more powerful than that.

I’ve used off the shelf APM products that don’t do as good of a job really isolating down to the method logging, and those come with a monthly subscription and a slow, lag ridden portal to boot. I think the bring-your-existing-logging-framework is one of the most powerful aspects of PostSharp, just being able to use what you already have, but supercharge those logs along the way.


This is a sponsored post however all opinions are mine and mine alone. 

ENJOY THIS POST?
Join over 3.000 subscribers who are receiving our weekly post digest, a roundup of this weeks blog posts.
We hate spam. Your email address will not be sold or shared with anyone else.

The most popular method of managing Azure resources in a programmatic fashion is Azure Resource Management Templates – or ARM templates for short. Much like Terraform, it’s a desired state type tool that you can define what you need, but Azure will work out the actual details of how to make it so (For the most part anyway!).

Over the years, I’ve ran into a few gotchas with these templates that I seem to forget and run into time and time again. Things that on the surface should be simple, but actually are confusing as hell. Often I end up googling the same issue every 3 months when I run into it again. So rather than do a post for each of these, I thought, why not combine them all together in a somewhat cheatsheet. If I’m having to constantly look these up, maybe you are too!

For now, I’ve named this “3 annoying gotchas”, but I’m likely to come back and edit this so maybe by the time you read this, we will be a little higher!

Let’s get started!

You Need To “Concat” A Database Connection String

In my ARM templates, I typically spin up an Azure SQL Database and a Keyvault instance. I make the Keyvault instance rely on the SQL Database, and immediately take the connection string and push it into keyvault. I do this so that there is never a human interaction that sees the connection string, it’s just used inside the ARM template, and straight into Keyvault.

But there’s an annoying gotcha of course! How do you get the connection string of an Azure SQL database in an ARM Template? You can’t! (Really, you can’t!). Instead you need to use string concatenation to build your connection string for storage.

As an example (And note, this is heavily edited, but should give you some idea) :

{
    "parameters" : {
        "sqlPassword" : {
            "type" : "securestring"
        }
    }, 
    ....
    "variables": {
        "sqlServerName": "MySQLServerName", 
        "sqlDbName" : "MySqlDatabase"
    }, 
    ....
    {
      "type": "Microsoft.KeyVault/vaults/secrets",
      "name": "MyVault/SQLConnectionString",
      "apiVersion": "2018-02-14",
      "location": "[resourceGroup().location]",
      "properties": {
        "value": "[concat('Server=tcp:',reference(variables('sqlserverName')).fullyQualifiedDomainName,',1433;Initial Catalog=',variables('sqlDbName'),';Persist Security Info=False;User ID=',reference(variables('sqlserverName')).administratorLogin,';Password=',parameters('sqlPassword'),';Connection Timeout=30;')]"
      }
    },
}

Or if we pull out just the part that is creating our SQL Connection String :

[concat('Server=tcp:',reference(variables('sqlserverName')).fullyQualifiedDomainName,',1433;Initial Catalog=',variables('sqlDbName'),';Persist Security Info=False;User ID=',reference(variables('sqlserverName')).administratorLogin,';Password=',parameters('sqlPassword'),';Connection Timeout=30;')]

So why do we have to go to all of this hassle just to get a connection string? There’s actually two reasons :

  • A connection string may have additional configuration, such as a timeout value. So it’s usually better that you get the connection string exactly how you need it.
  • But the most important reason is that a SQL Password, when set in Azure, is a blackbox. There is no retrieving it. You can only reset it. So from the ARM Templates point of view, it can’t ask for the connection string of a SQL database because it would never be able to get the password.

On that last note, it’s why when you try and grab your connection string from the Azure portal, it comes with a {your_password} field where your password will be.

Connecting Web Apps/Functions To Application Insights Only Requires The Instrumentation Key

I talked about this a little in a previous post around connecting Azure Functions to App Insights. I think it could be a hold over from the early days of App Insights when there wasn’t as much magic going on, and you really did have to do a bit of work to wire up Web Applications to App Insights. However now, it’s as simple as adding the Instrumentation Key as an app setting and calling it a day.

For example :

{
  "name": "APPINSIGHTS_INSTRUMENTATIONKEY",
  "value": "[reference(resourceId('Microsoft.Insights/components', variables('AppInsightsName')), '2014-04-01').InstrumentationKey]"
}

Also notice in this case, we can get the entire instrumentation key via the ARM template. I want to point this out because I’ve seen people manually create the Application Insights instance, then loop back around and run the ARM template with the key as an input parameter. You don’t have to do this! You can grab it right there in the template.

And again, as long as you use the appsetting name of “APPINSIGHT_INSTRUMENTATIONKEY” on either your Web Application or Azure Function, you are good to go!

Parameters File Cannot Contain Template Expressions

There are many times where you read a tutorial that uses a parameters file with a keyvault reference.

As an example, consider the following parameters file :

"parameters": {
    "serviceBusName": {
        "reference": {
            "keyVault": {
                "id": "/subscriptions/GUID/resourceGroups/KeyVaultRG/providers/Microsoft.KeyVault/vaults/KeyVault"
            },
        "secretName": "serviceBusName"
        }
    }
}

The idea behind this is that for the parameter of serviceBusName, we should go to keyvault to find that value. However, there’s something very wrong with this. We have a hardcoded subscription and resource group name. It makes far more sense for these to be dynamic, because between Dev, Test and Prod, we may have different subscriptions and/or resource groups right?

So, you may think this could be solved like so :

"parameters": {
    "serviceBusName": {
        "reference": {
            "keyVault": {
                "id": "[resourceId(subscription().subscriptionId, resourcegroup().name, 'Microsoft.KeyVault/vaults', parameters('KeyVaultName'))])"
            },
        "secretName": "serviceBusName"
        }
    }
}

But unfortunately :

resourceId function cannot be used while referencing parameters

You cannot use the resourceId function, or really any template expressions (Not even concat), inside a parameters file. It’s static text only. What that means is, frankly, that references to keyvault from a parameters file is pointless. In no situation have I ever wanted a hardcoded subscription ID in an ARM template, it just wouldn’t happen.

Microsoft’s solution for this is to push for the use of nested templates. In my personal view, this adds a tonne of complexity, but it’s an option. What I generally end up doing is trying to avoid Keyvault secrets at all. Usually my C# application is talking to keyvault anyway so there is no need for additional parameters like the above.

In anycase, the actual point of this section is to say that a parameters file cannot be dynamic without using nested templates. Whether that be for keyvault references or something else, you’ll have to find a way around using dynamic parameters.

ENJOY THIS POST?
Join over 3.000 subscribers who are receiving our weekly post digest, a roundup of this weeks blog posts.
We hate spam. Your email address will not be sold or shared with anyone else.

I’ve recently been doing battle trying to get Azure Application Insights playing nice with an Azure Function. Because they are from the same family I thought there wouldn’t be an issue but, Microsoft’s lack of documentation is really letting down the team here. This will be a short and sweet post that hopefully clears some things up.

Adding Application Insights

So the first thing that is different about using Application Insights with an Azure Function is that you don’t need any additional nuget packages. Under the hood, the packages that a function relies on out of the box themselves rely on the application insights package. So theoretically, everything is set up for you.

The only thing you actually need to do is set an application key of “APPINSIGHTS_INSTRUMENTATIONKEY” somewhere in your application.

For a function hosted on Azure, this is easy, you can do this on the configuration tab of your function and add your instrumentation key there.

Locally, you will be using either local.settings.json or appsettings.json depending on how your function is set up. Generally, either will work but it mostly depends on your individual project how you are managing settings locally.

Again, you don’t need to do anything to read this key, you just need to have it there and automagically, the function will wire everything up.

Now the other thing to note is that in the Azure Portal, on a Function, you’ll have an option to “Enable Application Insights” if you haven’t already. It looks a bit like so :

But actually all this does is add the instrumentation key to your appsettings. Just like we do above. It doesn’t do any fancy behind the scenes wiring up. It’s literally just a text field that wires everything up for you.

Configuring Application Insights For Azure Functions

So the next thing I found was that you were supposedly able to edit your host.json file of your function, and add in settings for insights. But what I found is that there is a tonne of settings that aren’t documented (yet?). The official documentation is located here : https://docs.microsoft.com/en-us/azure/azure-functions/functions-host-json. It looks good, but doesn’t seem to to have quite as many options for Application Insights as say, using it in a regular C# app.

So I actually had to dig into the source code. That took me here : https://github.com/Azure/azure-webjobs-sdk/blob/v3.0.26/src/Microsoft.Azure.WebJobs.Logging.ApplicationInsights/ApplicationInsightsLoggerOptions.cs. These are the actual settings that you can configure, some of which you cannot find documentation for but can make some educated guesses on what they do.

For me, I needed this :

"dependencyTrackingOptions": {
    "enableSqlCommandTextInstrumentation" :  true
}

This enables Application Insights to not only capture that a SQL command took place, but capture the actual text of the SQL so that I can debug any slow queries I see happening inside the application.

Again, I couldn’t find any documentation on setting this variable up, except the original source code. Yay open source!

If It Doesn’t Work, Chances Are There Is A Bug

The other thing I noticed about Application Insights in general is that there are a tonne of bugs that hang around for much longer than you might expect. For example, when I first added my app insights key to my function, I wasn’t collecting any information about SQL queries coming from the app. Asking around, people just assumed maybe you had to add another nuget package for that, or that I had set something up wrong.

Infact, there is a bug that has been 3 – 6 months that certain versions of EntityFramework suddenly don’t work with App Insights. Insights would capture the correct request, but it wouldn’t log any SQL dependency telemetry with any version of EFCore above 3.1.4.

https://stackoverflow.com/questions/63053334/enable-sql-dependency-in-application-insights-on-azure-functions-with-ef-core
https://github.com/microsoft/ApplicationInsights-dotnet/issues/2032
https://github.com/Azure/Azure-Functions/issues/1613

How does this help you? Well it probably doesn’t unless specifically you are missing SQL queries from your App Insights. But I just want to point out that by default, out of the box, adding Application Insights to an Azure Function should capture *everything*. You do not have to do anything extra. If you are not capturing something (For example, I saw another bug that it wasn’t capturing HttpClient requests correctly), then almost certainly it will be the mishmash of versions of something you are using causing the problem.

ENJOY THIS POST?
Join over 3.000 subscribers who are receiving our weekly post digest, a roundup of this weeks blog posts.
We hate spam. Your email address will not be sold or shared with anyone else.

Since really .NET Framework 1, the ability for .NET Console apps to parse command line flags and actually provide helpful feedback to the user on even the availability of such flags has been severely lacking.

What do I mean by that? Well when you create a new console application in C#/.NET/.NET Core, your code will be given a simple array of string arguments. These won’t be filtered in any way and will basically just be given to you wholesale. From there, it’s up to you to create your own level of boilerplate to parse them out, run any validation you need to, *then* finally get on to actually creating the logic for your app :

static int Main(string[] args)
{
    //Boilerplate for parsing the args array goes here
}

And it’s not like out of the box, someone running the console application can get helpful feedback on the flags either. If you compare that to say a simple “dotnet” command. Running it without any flags gives you atleast some helpful information on possible options to get things up and running.

C:\Users\wadeg> dotnet

Usage: dotnet [options]
Usage: dotnet [path-to-application]

Options:
  -h|--help         Display help.
  --info            Display .NET information.
  --list-sdks       Display the installed SDKs.
  --list-runtimes   Display the installed runtimes.

path-to-application:
  The path to an application .dll file to execute.

But all that’s about to change with Microsoft’s new library called System.CommandLine!

Creating A Simple Console App The Old Fashioned Way

Before we go digging into the new goodies. Let’s take a look at how we might implement a simple console application parsing the string args ourselves.

Here’s a console application I created earlier that simply greets a user with their given name, title, and will change the greeting depending on if we pass in a flag saying it’s the evening.

static int Main(string[] args)
{
    string name = string.Empty;
    string title = string.Empty;
    bool isEvening = false;

    for (int i = 0; i < args.Length; i++)
    {
        var arg = args[i].ToLower();
        if (arg == "--name")
        {
            name = args[i + 1];
        }

        if (arg == "--title")
        {
            title = args[i + 1];
        }

        if (arg == "--isevening")
        {
            isEvening = true;
        }
    }

    if (string.IsNullOrEmpty(name))
    {
        Console.WriteLine("--name is a required flag");
        return -1;
    }

    var greeting = isEvening ? "Good evening " : "Good day ";
    greeting += string.IsNullOrEmpty(title) ? string.Empty : title + " ";
    greeting += name;
    Console.WriteLine(greeting);

    return 0;
}

The code is actually quite simple, but let’s take a look at it bit by bit.

I’ve had to create a sort of loop over the args to work out which ones were actually passed in by the user, and which ones weren’t. Because the default args doesn’t actually distinguish between what’s a flag and what’s a passed in parameter value, this is actually quite messy.

I’ve also had to write my own little validator for the “–name” flag because I want this to be mandatory. But there’s a small problem with this..

How can a user know that the name flag is mandatory other than trial and error? Really they can’t. They would likely run the application once, have it fail, and then add name to try again. And for our other flags, how does a user know that these are even an option? We would have to rely on us writing good documentation and hope that the user reads it before running (Very unlikely these days!).

There really isn’t any inbuilt help with this application, we could try and implement something that if a user passed in a –help flag, we would return some static text to help them work out how everything runs, but this isn’t self documenting and would need to be updated each time a flag is updated, removed or added.

The reality is that in most cases, this sort of helpful documentation is not created. And in some ways, it’s relegated C# console applications to be some sort of quick and dirty application you build for other power users, but not for a general everyday developer.

Adding System.CommandLine

System.CommandLine is actually in beta right now. To install the current beta in your application you would need to run the following from your Package Manager Console

Install-Package System.CommandLine -Version 2.0.0-beta1.20574.7

Or alternatively if you’re trying to view it via the Nuget Browser in Visual Studio, ensure you have “Include prerelease” ticked.

Of course by the time you are reading this, it may have just been released and you can ignore all that hassle and just install it like you would any other Nuget package!

I added the nuget package into my small little greeter application, and rejigged the code like so :

static int Main(string[] args)
{
    var nameOption = new Option(
            "--name",
            description: "The person's name we are greeting"
        );
    nameOption.IsRequired = true;

    var rootCommand = new RootCommand
    {
        nameOption, 
        new Option(
            "--title",
            description: "The official title of the person we are greeting"
        ),
        new Option(
            "--isevening",
            description: "Is it evening?"
        )
    };
    rootCommand.Description = "A simple app to greet visitors";

    rootCommand.Handler = CommandHandler.Create<string, string, bool>((name, title, isEvening) =>
    {
        var greeting = isEvening ? "Good evening " : "Good day ";
        greeting += string.IsNullOrEmpty(title) ? string.Empty : title + " ";
        greeting += name;
        Console.WriteLine(greeting);
    });

    return rootCommand.Invoke(args);
}

Let’s work through this.

Unfortunately, for some reason the ability to make an option “required” cannot be done through an option constructor, hence why our first option for –name has been setup outside our root command. But again, your mileage may vary as this may be added before the final release (And it makes sense, this is probably going to be a pretty common requirement to make things as mandatory).

For the general setup of our flags in code, it’s actually pretty simple. We say what the flag name is, a description, and we can even give it a type right off the bat so that it will be parsed before getting to our code.

We are also able to add a description to our application which I’ll show shortly why this is important.

And finally, we can add a handler to our command. The logic within this handler is exactly the same as our previous application, but everything has been set up for us and passed in.

Before we run everything, what happens if we just say run the application with absolutely no flags passed in.

Option '--name' is required.

CommandLineExample:
  A simple app to greet visitors

Usage:
  CommandLineExample [options]

Options:
  --name <name> (REQUIRED)    The person's name we are greeting
  --title <title>             The official title of the person we are greeting
  --isevening                 Is it evening?
  --version                   Show version information
  -?, -h, --help              Show help and usage information

Wow! Not only has our required field thrown up an error, but we’ve even been given the full gamut of flags available to us. We’ve got our application description, each flag, and each flags description of what it’s intended to do. If we run our application with the –help flag, we would see something similar too!

Of course there’s only one thing left to do

CommandLineExample.exe --name Wade
Good Day Wade

Pretty powerful stuff! I can absolutely see this becoming part of the standard .NET Core Console Application template. There would almost be no reason to not use it from now on. At the very least, I could see it becoming a checkbox when you create a Console Application inside Visual Studio to say if you want “Advanced Arguments Management” or similar, it really is that good!

ENJOY THIS POST?
Join over 3.000 subscribers who are receiving our weekly post digest, a roundup of this weeks blog posts.
We hate spam. Your email address will not be sold or shared with anyone else.

An XML External Entity vulnerability (Or XXE for short) is a type of vulnerability that exploits weaknesses (Or more so features) in how external entities are loaded when parsing XML in code. Of course, OWASP has a great guide on it here, but in it’s most basic form, we can trick code into loading an external resource (Either a file on the target machine, or even a remote page on the same network) and giving us that information in some way.

For example, consider an ecommerce application allows you to update a production description by submitting the following XML to the server :

<product id="1">
    <description>What a great product!</description>
</product>

Then consider the following payload :

<!DOCTYPE foo [ <!ENTITY xxe SYSTEM "file:///etc/passwd"> ]>
<product id="1">
    <description>&xxe;</description>
</product>

That may look confusing but essentially what we are doing is creating an internal variable called “xxe”, and storing the contents of the local password file (on linux) into it. Then we are setting the production description to that variable. Once completed, our production description will now leak all of the systems passwords.

It’s not just local files either, if a machine has access to internal only websites, then this could also be leveraged :

<!DOCTYPE foo [ <!ENTITY xxe SYSTEM "http://someinternalwebsite"> ]>
<product id="1">
    <description>&xxe;</description>
</product>

Not many people realize that many XML parsers have the “feature” to reach out and load external entities and pull them into the XML, but very clearly, it’s a huge security risk. So much so that in 2020, XXE attacks were ranked number 4 in OWASP’s top 10 web application security list. Ouch!

Testing XXE In .NET Core

So it got me thinking for .NET Core, how could I test under what circumstances XXE can actually occur. After all, like SQL Injection, I always hear people say “Well that’s not relevant anymore, the framework protects you”. But does it really? And even if it does by default, how easy is it to shoot yourself in the foot?

My first step was to setup a testing rig to try out various pieces of code and see if they fit. It was actually rather simple. First I created a static class that allowed me to pass in a method that parses XML, and then I could validate whether that method was safe or not.

public static class AssertXXE
{
    private static string _xml = "<!DOCTYPE foo [<!ENTITY xxe SYSTEM \"_EXTERNAL_FILE_\">]> <product id=\"1\"> <description>&xxe;</description></product>";

    public static void IsXMLParserSafe(Func<string, string> xmlParser, bool expectedToBeSafe)
    {
        var externalFilePath = Path.GetFullPath("external.txt");
        var xml = _xml.Replace("_EXTERNAL_FILE_", externalFilePath);
        var parsedXml = xmlParser(xml);

        var containsXXE = parsedXml.Contains("XXEVULNERABLE");

        Assert.AreEqual(containsXXE, !expectedToBeSafe);
    }
}

You may ask why I should pass in a boolean as to whether something is safe or not. I debated this. When I find an unsafe way of parsing XML, I didn’t want the test to “fail” per say. Because it became hard to figure out which were failing because they *should* fail, and which ones should fail because I made a simple syntax error. This way, once I found a vulnerable way of loading XML, I could then simply mark it that in future, I expect it to always be unsafe.

Onto the actual tests themselves, they were pretty simple like so :

[Test]
public void XmlDocument_WithDefaults_Safe()
{
    AssertXXE.IsXMLParserSafe((string xml) =>
    {
        var xmlDocument = new XmlDocument();
        xmlDocument.LoadXml(xml);
        return xmlDocument.InnerText;
    }, true);
}

And so on. But onto the actual results…

Testing XmlDocument

The XmlDocument type in C# is “mostly” safe. Talking strictly .NET Framework 4.5.2 onwards (Including into .NET Core), the default setup of an XML Document was safe. So for example, this is not a vulnerable test :

[Test]
public void XmlDocument_WithDefaults_Safe()
{
    AssertXXE.IsXMLParserSafe((string xml) =>
    {
        var xmlDocument = new XmlDocument();
        xmlDocument.LoadXml(xml);
        return xmlDocument.InnerText;
    }, true);
}

However, providing an XMLResolver to your XMLDocument made it eager to please and would download external entities. So this for example, would be unsafe :

var xmlDocument = new XmlDocument();
xmlDocument.XmlResolver = new XmlUrlResolver(); //<-- This!
xmlDocument.LoadXml(xml);
return xmlDocument.InnerText;

Remember how I mentioned that .NET Framework 4.5.2 > was safe? That’s because from that point, the XMLResolver was defaulted to null whereas earlier versions had a default resolver already set with the default XmlDocument constructor.

But for my use case, using XmlDocument in .NET Core with the defaults is not vulnerable to XXE.

Testing XmlReader

Next I took a look at XmlReader. Generally speaking, you can tie in an XmlReader to read a document, but then parse on any manipulation to a second class. So what I wanted to test was if I was using an XmlReader, and passing it to an XmlDocument class that was vulnerable, could the reader stop the disaster before it even got to the XmlDocument?

The answer was yes! Setting DtdProcessing to Prohibit would actually throw an exhibition when parsing the XML, and not allow processing to continue. Prohibit is also the default behaviour which was great!

XmlReaderSettings settings = new XmlReaderSettings();
settings.DtdProcessing = DtdProcessing.Prohibit;
settings.MaxCharactersFromEntities = 6000;

using (MemoryStream stream = new MemoryStream(Encoding.UTF8.GetBytes(xml)))
{
    XmlReader reader = XmlReader.Create(stream, settings);

    var xmlDocument = new XmlDocument();
    xmlDocument.XmlResolver = new XmlUrlResolver();
    xmlDocument.Load(reader);
    return xmlDocument.InnerText;
}

This also held true if I set DtdProcessing to ignore like so :

settings.DtdProcessing = DtdProcessing.Ignore;

Although I would get the following exception because instead of simply stopping parsing, it would still try and parse the document, but ignore all entity declarations.

Reference to undeclared entity 'xxe'.

Interestingly, to make XmlReader unsafe I had to do two things. First, I have to make DtdProcessing be set to “Parse” *and* I had to set a UrlResolver up :

XmlReaderSettings settings = new XmlReaderSettings();
settings.DtdProcessing = DtdProcessing.Parse;
settings.XmlResolver = new XmlUrlResolver();

Without these settings on the reader, even if the resulting stream was passed to an XmlDocument with a Resolver setup, it was still not vulnerable.

Getting Involved

For my particular use cases, what I found was that the way in which I use XmlDocument in .NET Core was safe. I never manually set an XmlResolver up, so I was good to go. But maybe you’re using a different way to parse XML? Maybe you’re even using a third party library to work with XML?

For this, I’ve thrown up my code that I used to test my scenarios on Github. You can access it here : https://github.com/mindingdata/XXEDotNetCore

If you, or the company you work for parse XML a different way, I really encourage you to add a PR on whether it is safe or unsafe for XXE. Again, this harks back to what I said earlier that so many of these OWASP top 10 security issues, developers like to say “Oh, that’s an old thing, it’s not a problem anymore”. And maybe for the majority of use cases that’s true, but it really doesn’t hurt to rig up your code and actually prove that’s the case!

ENJOY THIS POST?
Join over 3.000 subscribers who are receiving our weekly post digest, a roundup of this weeks blog posts.
We hate spam. Your email address will not be sold or shared with anyone else.

I’ve recently been diving into the new Channel type in .NET Core, and something I’ve noticed time and time again is how much effort goes into making sure the entire type is threadsafe. That is, if two threads are trying to act on the same object, they are synchronized one after the other instead of just being a free for all. In Microsoft’s case with Channel<T>, they use a combination of the lock keyword, async tasks, and a “queue” to obtain locks.

It somewhat belies belief that at the end of the day, to call something “threadsafe”, you have to write 100’s of lines of code that don’t actually provide any function except trying to make sure you don’t shoot yourself in the foot with a simple multithreaded scenario. And then there’s the fact that if you get it wrong, you probably won’t know until weird errors start appearing in your production logs that you can never seem to reproduce in development because you haven’t been able to hit the race condition lottery.

And then I came across the Postsharp Threading Library

PostSharp Multithreading Library

To be honest, they had me from the moment I read this beauty of a tag line :

Write verifiable thread-safe code in .NET without your brain exploding with PostSharp Threading

Sounds good to me!

PostSharp Threading is actually part of an entire suite of libraries from PostSharp that work on removing common boilerplate scenarios that I’m almost certain every .NET Developer has run into before. They have solutions for caching, logging, MVVM, and of course, threading. For today, I’m just going to focus on the threading library as that’s been boggling my mind for the past couple of weeks. Let’s jump right in!

Using Locks To Synchronize Multithreaded Data Access In C#

I want to give a dead simply way in which you can wrap yourself in knots with multithreading that both the compiler and the runtime may not make you aware of at first (If ever). Take the example code :

class Program
{
    static void Main(string[] args)
    {
        MyClass myClass = new MyClass();
        List<Task> tasks = new List<Task>();

        for(int i=0; i < 100; i++)
        {
            tasks.Add(
                Task.Run(() =>
                {
                    for (int x = 0; x < 100; x++)
                    {
                        myClass.AddMyValue();
                    }
                })
            );
        }

        Task.WaitAll(tasks.ToArray());

        Console.WriteLine(myClass.GetMyValue());
    }
}

class MyClass
{
    private int myValue = 0;

    public void AddMyValue()
    {
        myValue++;
    }

    public int GetMyValue()
    {
        return myValue;
    }
}

Hopefully it’s not too confusing. But let’s talk about some points :

  1. I have a class called “MyClass” that has an integer value, and a method to add 1 to the value.
  2. In my main method, I start 100 threads (!!!) and all these threads do is loop 100 times, adding 1 to the value of myClass.
  3. myClass is shared, so each thread is accessing the same object.
  4. I wait until the threads are all finished.
  5. Then I output the value of myClass.

Any guesses what the output of this program will be? Thinking logically, 100 threads, looping 100 times, we should see the application output 10000. Well I ran this little application 5 times and recorded the results.

6104
8971
9043
9256
8833

Oof, what’s going on here? We have a classic multithreading issue. Two (or more) threads are trying to update a value at the same time, resulting in us getting a complete meltdown when it comes to actually incrementing our value.

So how would we solve this *without* PostSharp threading?

At first it actually seems quite simple, we simply wrap our increment in a lock like so :

public void AddMyValue()
{
    lock (this)
    {
        myValue++;
    }
}

If we run our application now..

10000

Perfect!

But there are some downsides to this, and both are issues with maintainability.

  1. What if we have multiple methods in our class? And multiple classes? We now need to spend an afternoon adding locks to all methods.
  2. What if a new developer comes along, and adds a new method? How do they know that this class is used in multithreaded scenarios requiring locks? Same goes for yourself. You need to remember to wrap *every* method in locks now if you want to keep this class threadsafe! You very easily could have a brain fade moment, not realize that you need to add locks, and then only once things hit production do you start seeing weird problems.

Using The PostSharp Synchronized Attribute

So how can PostSharp help us? Well all we do is add the following nuget package :

Add-Package PostSharp.Patterns.Threading

Then we can modify our class like so :

[Synchronized]
class MyClass
{
    private int myValue = 0;

    public void AddMyValue()
    {
        myValue++;
    }

    public int GetMyValue()
    {
        return myValue;
    }
}

Notice all we did was add the [Synchronized] attribute to our class and nothing else. This attribute automatically wraps all our methods in a lock statement, making them threadsafe. If we run our code again, we get the same correct result, same as using locks,  but without having to modify every single method, and without having to remember to add locks when a new method is added to the class.

You might expect some big long speel here about how all of this works behind the scenes, but seriously.. It. Just. Works. 

Using A Reader/Writer Model For Multithreaded Access

In our previous example, we used the Synchronized attribute to wrap all of our class methods in locks. But what about if some of them are actually safe to read concurrently? Take the following code example :

class Program
{
    static void Main(string[] args)
    {
        MyClass myClass = new MyClass();
        List tasks = new List();

        for(int i=0; i < 100; i++) { tasks.Add( Task.Run(() =>
                {
                    for (int x = 0; x < 100; x++)
                    {
                        myClass.AddMyValue();
                    }
                })
            );
        }

        Task.WaitAll(tasks.ToArray());

        //Now kick off 10 threads to read the value 10 times (Asd an example!)
        tasks.Clear();

        for(int i=0; i < 10; i++) { tasks.Add(Task.Run(() => { var myValue = myClass.GetMyValue(); }));
        }

        Task.WaitAll(tasks.ToArray());

    }
}

[Synchronized]
class MyClass
{
    private int myValue { get; set; }

    public void AddMyValue()
    {
        myValue++;
    }

    public int GetMyValue()
    {
        //Block the thread by sleeping for 1 second. 
        //This is just to simulate us actually doing work. 
        Thread.Sleep(1000);
        return myValue;
    }
}

I know this is a pretty big example but it should be relatively easy to follow as it’s just an extension of our last example.

In this example, we are incrementing the value in a set of threads, then we kick off 10 readers to read the value back to us. When we run this app, we may expect it to complete in roughly 1 second. After all, the only delay is that in our GetMyValue method, there is a sleep of 1000ms. However, these are all on Tasks so we should expect them to all complete roughly at the same time.

However, clearly we have also marked the class as Synchronized and that applies a lock to *all* methods, even ones that we are fairly certain won’t have issues being threadsafe. In our example, there is no danger in allowing GetMyValue() to run across multiple threads at the same time. This is quite commonly referred to as a Reader/Writer problem, that is generally solved by a “Reader/Writer Lock”.

The concept of a Reader/Writer lock can be simplified to the following :

  1. We will allow any number of readers concurrent access to read methods without blocking each other.
  2. A writer requires exclusive lock (Including blocking readers), until the writer is completed, then either all readers or another writer can gain access to the object.

This works perfect for us because at the end of our application, we want to allow all readers to have access to the value at once without blocking each other. So how can we achieve that? Actually it’s pretty simple!

[ReaderWriterSynchronized]
class MyClass
{
    private int myValue { get; set; }

    [Writer]
    public void AddMyValue()
    {
        myValue++;
    }

    [Reader]
    public int GetMyValue()
    {
        //Block the thread by sleeping for 1 second. 
        //This is just to simulate us actually doing work. 
        Thread.Sleep(1000);
        return myValue;
    }
}

We change our Synchronized attribute to a “ReaderWriterSynchronized”, we then go through and we mark each method noting whether it is a writer (So requires exclusive access), or a reader (Allows concurrent access).

Running our application again, we can now see it completes in 1 second as opposed to 10 as it’s now allowing GetMyValue() to be run concurrently across threads. Perfect!

Solving WPF/Winform UI Thread Updating Issues

I almost exclusively work with web applications these days, but I can still remember the days of trying to do multithreading on both Winform and WPF applications. If you’ve ever tried it, how often have you run into the following exception :

System.InvalidOperationException: Cross-thread operation not valid: Control ‘labelStatus’ accessed from a thread other than the thread it was created on.

It can be from something as simple as so in a Winform App :

private void buttonUpdate_Click(object sender, EventArgs e)
{
    Task.Run(() => UpdateStatus("Update"));
}

private void UpdateStatus(string text)
{
    try
    {
        labelStatus.Text = text;
    }catch(Exception ex)
    {
        MessageBox.Show(ex.ToString());
    }
}

Note that the whole try/catch with a MessageBox is just so that the exception is actually shown without the Task swallowing the exception. Otherwise in some cases we may not even see the exception at all, instead it just silently fails and we don’t see the label text update and wonder what the heck is going on.

The issue is quite simple. In both Winform and WPF, the controls can only be updated from the “UI Thread”. So any background thread (Whether a thread, task or background worker) needs to sort of negotiate the update back into main UI thread. For WinForms, we can use delegates with Invoke, and for WPF/XAML, we have to use the Dispatcher class. But both require us to write an ungodly amount of code just to do something as simple as update a label.

I would also note that sometimes you see people recommend adding the following line of code somewhere in your application :

CheckForIllegalCrossThreadCalls = false;

This is a terrible idea and you should never do it. This is basically hiding the error from you but the problem of two threads simultaneously trying to update/use a control still exists!

So how does PostSharp resolve this?

[Dispatched]
private void UpdateStatus(string text)
{
    try
    {
        labelStatus.Text = text;
    }catch(Exception ex)
    {
        MessageBox.Show(ex.ToString());
    }
}

With literally *one* attribute of course. You simply mark which methods need to be ran on the UI thread, and that’s it! And let me just say one thing, while yes at some point in your C# career you need to do a deep dive on delegates/actions and marshalling calls, I really wish I had this early on in my developer life so I didn’t have to spend hours upon hours writing boilerplate code just to update a label or change the color of a textbox!

Who Is This Library For?

I think if your code is kicking off tasks at any point (Especially if you are doing background work in a Winform/WPF environment), then I think giving PostSharp Threading a try is a no brainer. There is actually even more features in the library than I have listed here including a way to make objects immutable, freeze objects, and even be able to mark objects as unsafe for multithreading just to stop a future developer shooting themselves in the foot.

Give it a try and drop a comment below on how you got on.


This is a sponsored post however all opinions are mine and mine alone. 

ENJOY THIS POST?
Join over 3.000 subscribers who are receiving our weekly post digest, a roundup of this weeks blog posts.
We hate spam. Your email address will not be sold or shared with anyone else.