Lately I seem to have run into a tonne of these types of errors when trying to host .NET Core applications inside IIS
HTTP Error 502.5 - ANCM Out-Of-Process Startup Failure
and
HTTP Error 500.30 - ANCM In-Process Start Failure
And to be honest, they have been incredibly hard to debug. If you search for these errors on Google, you’ll find hundreds of stack overflow posts detailing people running into them, often with very little follow up on how they solved the issue. Sometimes they’ll even go as far to say “I re-installed everything and now it works” which in most cases, is not going to be viable. “Hey boss, let me just wipe this server real quick” generally doesn’t cut it.
But hopefully, fingers crossed, this will be the last blog post you’ll ever have to read on these error messages!
Why Are There Two Error Messages?
So first let’s touch on why there are two different messages (And error codes). Essentially, they are the exact same error but refer to different hosting models when running IIS infront of .NET Core. We won’t dive too deep into what these hosting models are, as at some point (.NET Core 2.1+ I believe) the default hosting model got swapped from Out-Of-Process to In-Process, so it very well could depend entirely on the version of .NET Core you are running.
Just know that for all practical purposes, these two error messages are the same so if you are following some blog post that is talking about one of these error messages, but you have the other, you can still follow along as it may well solve your problem.
Solution #1 – Incorrect Or Non-Existent .NET Core Hosting Bundle
So if you’re using something like Azure App Services that abstract away all the server management for you, then you can probably skip this, but if you are managing the server yourself (Or you’re trying to run a .NET Core application on your PC), then definitely read on.
If you followed our guide to running .NET Core on IIS (Which you definitely should) you would know that you need to install the .NET Core “Hosting Bundle” for .NET Core web apps to work inside IIS. This is also required even if you use self contained deployments. It is also required even if you install the .NET Core runtime on the machine. I’ll repeat, you cannot host .NET Core web applications inside IIS unless you install the hosting bundle.
If you are on your server and you aren’t sure if you have the hosting bundle installed, go to Add/Remove Programs in Windows and search for hosting :
If you don’t have this, then you can go here : https://dotnet.microsoft.com/download/dotnet-core/3.1 and download the hosting bundle :
It’s also important to note. Download the hosting bundle that matches the version of .NET Core you are running. I would presume that things are backwards compatible and if you download a newer version, it will work with older versions of .NET Core, but it’s super important to have atleast the same version as your actual code.
And finally. Restart your machine. I cannot make that any more clear. I cannot tell you how many people install everything under the sun and nothing seems to “work”, but it’s because they haven’t restarted. For the most part, you can get away with just an IISReset, but restarting the machine is often easier and just makes sure that you are restarting absolutely everything you need to.
For Shared Hosting, you may need to open a ticket with your provider. Again, I’ve seen providers only have the .NET Core 2 Hosting Bundle and people are trying to deploy .NET Core 3.1 applications. It pays to ask!
Still not working? Move onto Solution #2.
Solution #2 – .NET Core Startup Process Fails
This one took me an absolute lifetime to workout, but as it turns out, the .NET Core startup process has to start successfully before IIS can take over. That means if certain startup routines of .NET Core fail, then IIS doesn’t know how to handle the exception and just gives a generic error. Infact, that’s why it’s called an “ANCM Startup Failure”!
So what sort of things are we talking? Well for me, the most common one has been integrating with KeyVault. If you follow the official documentation here, it tells you to add it to your CreateHostBuilder method. CreateHostBuilder is ran on startup and needs to complete *before* IIS can take over, if for some reason your app cannot connect to KeyVault (Firewall, permissions etc), then your applicaiton will crash and IIS will have no option but to bail out as well.
Generally speaking, most of the times I see these ANCM startup failures has been when I’ve added code to the CreateHostBuilder method in my program.cs. Truth be told, it’s actually 100% of the time.
But how can we diagnose them? The easiest way is to actually run our application via Kestrel direct on the server. On a metal server (e.g. One you can RDP to), this means simply going to the folder where our application dlls are, and running “dotnet myapplication.dll” and seeing if it runs. What you should see is that the startup error is printed to the console and should point you in the right direction. Remember, in production configuration, firewalls, keys, and secrets could all be completely different so just because the application starts up fine on your local machine, on a remote machine it could completely blow up.
For sites hosted inside things like Azure App Services, you need to find a way to run the application manually yourself. On Azure, this means using Kudu, but on shared hosting services, you might be completely out of luck. If that’s you, you really have two options :
- Open a support ticket to see if they can run your application manually in Kestrel like above, and hand you the error. (Again, this might be worth it to see if they even support the version of .NET Core you are intending to run).
- Guess. Yes guess. Check your applications program.cs and how your HostBuilder is created. Start removing things and redeploying until you find the thing that is breaking it. If possible, create a simple hello world application and deploy it, if it also breaks, then it points more towards the hosting bundle than anything wrong with your startup code.
Debugging With stdout
Still need further debugging. Here’s another tip!
After publishing a .NET Core app, you will find in the publish directory there is a web.config with minimal information, but you should see something like so :
<aspNetCore processPath="dotnet" arguments=".\MyTestApplication.dll" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout" hostingModel="inprocess" />
Pretty obvious here, we just change stdoutLogEnabled=”false” to stdoutLogEnabled=”true” and redeploy. This should enable additional IIS logging to ./logs and allow us to debug further. Honestly, I’ve found this really really hit and miss. I’ve found that running the application directly from the console on the server to be much more helpful, but this is also an option if you have no other way.
Still Not Working?
Still not working? Or solved it a different way? Drop a comment below with a link to a GIT Repo and let the community help. This is honestly an infuriating error and I’ve had 20 year veteran programmers pull their hair out trying to solve it, so seriously drop a line. If you aren’t comfortable leaving a comment to your code, try wade[at]dotnetcoretutorials.com and I’ll see what I can do!
