ASP.NET Core Correlation IDs Writing a basic middleware library to enable correlation IDs on ASP.NET Core

On a recent project we needed to implement the concept of correlation IDs across some ASP.NET Core API services. In our scenario we have an API service called from a front end JavaScript application which will then call one or more secondary back end API services to gather data. This is becoming a pretty common requirement when we have the concept of “microservices” which communicate over HTTP.

Why we need correlation IDs?

A problem that arises with having multiple services separated into distinct units is how we track the flow of a single user request through each of the individual services that might be involved in generating a response. This is particularly crucial for logging and diagnosing faults with the system as a whole. Once the request leaves the first API service and passes control onto the backend API service, we start to struggle to interpret the logs. A failure in the backend API will likely impact the front end API service but those might appear to be two unrelated errors. We need a way to view the entire flow from when the request hit the front end API, through to the backend API and back again.

Solution

This is where the concept of correlation IDs comes into play. A correlation ID is simply a unique identifier that is passed through the entire request flow and crucially is passed between the services. When each service needs to log something, it can include this correlation ID, thereby ensuring that we can track a full user request through from start to finish. We can parse all of the logs from each distinct part of the system, looking for the correlation ID. We can then form a full picture of the lifetime of that request and hopefully identify root causes of any errors.

In this post I’m going to discuss a very simple library that I wrote to help solve this problem. It’s a pretty simple implementation and fits specifically the requirement we needed to achieve. I have some thoughts about extending it to make working with it a bit simpler. I’d like to expose the correlation ID via some kind of context class that allows you to more easily access the ID without the dependency on IHttpContextAccessor in places where the HttpContext is not already exposed. For now, let’s look at how this initial version works.

I hope this will serve as a nice real-world example of how it’s quite simple to build and package a middleware library that you can re-use across your own projects.

Correlation ID Options

As part of the solution I wanted to provide a mechanism to configure the middleware and so I followed the standard pattern used in ASP.NET Core to provide a class representing the configuration properties. My middleware is pretty straightforward so only has two options.

There is an option to control the name of the request header where the correlation ID will be passed between services. Both the sending API and receiving API need to use the same header name to ensure they can locate and pass on the correlation ID. I default this to “X-Correlation-ID” which is a standard name that I found for this type of header.

The second option controls whether the correlation ID is included in the HttpResponse. I included this for our scenario so that the final front end API can pass the ID down to the front end SPA code. This allows us to tie up and client side logging with the backend logs. This feature is enabled by default but can be disabled in the configuration.

Middleware

The main part of the solution involves a piece of middleware and an extension method to provide an easy way to register the middleware into the pipeline. The structure of this code follows the patterns defined as part of ASP.NET Core. The middleware class looks like this:

The constructor accepts our configuration via the IOptions<T> concept which is part of the ASP.NET Core framework. We will use these options values to control the behaviour of the middleware.

The main Invoke method, which will be called by the framework, is where the “magic” happens.

First we check for an existing correlation ID coming in via the request headers. We check for a header matching the name configured in the options and if we find a matching value we set the TraceIdentifier property on the HttpContext. I take advantage of one of the improvements from C# 7 which means we can declare the out variable inline, rather than having to pre-declare it before the TryGetValue line.

If we do not find a correlation ID on the request header we just utilise the existing TraceIdentifier which is generated automatically.

The next block is optional, controlled by the IncludeInResponse configuration property. If true (the default) we use the OnStarting callback to allow us to safely include the correlation ID header in the response.

Extension Method

The final piece of code needed (although not absolutely required) is to include some extension methods to make adding the middleware to the pipeline easier for consumers. The code in that class looks like this:

It provides a UseCorrelationId method (and two overloads) which are extension methods on the IApplicationBuilder. These will register the CorrelationIdMiddleware and if provided handle taking a custom name for the header, or a complete CorrelationIdOptions object which will ensure the middleware behaves as expected.

With this in place, all that a consumer of the library needs to do is include it in their project and then call the appropriate UseCorrelationId method from the Configure method in the startup class.

Summary

Middleware presents a very easy way to build the logic required by the concept of correlation IDs into and ASP.NET Core application. With a few small components I was able to put together a basic solution to solve our immediate problem. This was my first time releasing my own library as open source and publishing a public Nuget package. I’ve decided to summarise the steps I took for that in my next blog post.

If you want to review the full source, you can check it out on GitHub.

If you want to pull in the NuGet package to your work, you can find it up on Nuget.org.

Getting started with ASP.NET Core 2.0 Preview 1 A quick tour of the main structural changes

At Microsoft Build 2017 yesterday, the formal announcement of ASP.NET Core 2.0 and .NET Core 2.0 was made.

ASP.NET Core 2.0 preview announcement

.NET Core 2.0 preview announcement

For me it was no surprise that we were close with these announcements as I keep an eye on the ASP.NET Core repositories on GitHub where there have been preview 1 branches for a little while. Additionally, the team have shared some information on the weekly ASP.NET Community Standup shows that have helped us understand some of the changes coming our way.

In this post I want to focus in on a few changes to the basic structure of ASP.NET Core 2.0 applications which simplify the code needed to get started. Before looking at those elements, I felt it would be worth sharing the steps I took to get started with ASP.NET Core 2.0 Preview 1 on my development machine.

Getting started with ASP.NET Core 2.0 preview 1

The first step was to get the preview SDK for .NET Core 2.0 which can be safely installed alongside any prior 1.x SDK versions. The announcement post provides a link to download the SDK.

The next step was to install the SDK. Nice and easy!

Installing .NET Core 2.0 SDK

Initially I tried creating a project in the existing Visual Studio 2017 IDE. I did so by starting with an ASP.NET Core 1.1 project and simply updating it to target netcoreapp2.0 and the latest ASP.NET Core 2.0 packages. However, this was problematic since I couldn’t restore or build the solution inside Visual Studio. I did manage to do both of those tasks at the command line level using dotnet restore and dotnet build. Finally, running dotnet run I was up and running with a small sample API site. 

After querying the lack of Visual Studio support on Twitter, Damian Edwards confirmed that I would need to use the preview version of Visual Studio 2017 in order to work with ASP.NET 2.0 preview projects. So I went off to download and then install it. According to their website this is safe to install side-by-side with other versions. If you want to follow along, download Visual Studio 2017 preview from here.

Installing Visual Studio 2017 preview

Now that I had VS 2017 preview installed on my machine I decided to start fresh and try creating a new project directly inside the IDE. The new project dialog for an ASP.NET Core web project has been updated so I could simply choose to target that from the UI.

ASP.NET Core 2.0 project template

After a few moments I was up and running, working on a new ASP.NET 2.0 web application. A very painless experience so well done and thanks to the team for the work they’ve done to get us started on this preview release.

A Quick Tour of ASP.NET Core 2.0 preview 1

Now that I have a default ASP.NET Core 2.0 project on my machine, I want to highlight a few of the initial changes that you’ll see.

Simplified CSPROJ

After the move back to csproj that came with Visual Studio 2017, the new structure for these project files has been a great improvement on the past. Gone are the ugly GUIDs and the requirement to specify all files. With this release, the team have simplified things a step further.

In the mainstream Visual Studio 2017 release, targeting ASP.NET Core 1.1 a default csproj file looks like this:

In the latest version, we now have a single meta-package we reference to bring in all of the ASP.NET Core packages. This reduces a default csproj to just these few lines:

At first this concerned me a little. A key message behind the original ASP.NET Core release, was a “pay to play” model. You could specify only the packages you actually needed in your application, no longer bringing a dependency on a whole monolith framework. You could bring each version of each component that you actually used. This also resulted in a smaller footprint for your published application. My first impression was that we were taking a bit of a step back by instead referencing a single master package.

However after reading the announcement and chatting on Twitter, the consensus seems to be that this is not a problem. The SDK ships with the packages included with it, so you are never pulling these from NuGet directly. That had been a concern for me since at work we build our projects inside Docker containers. If each new build needed to pull all of the packages, even those we didn’t need, across the Internet, it could slow down our builds.

However, since they are local with the SDK, this is not a problem. Then I had a concern about a large deployment when you publish your application, including binaries that my application doesn’t even use. This also seems to be solved, as Microsoft now have a package trimming feature that they explain will exclude the binaries you are not using for your publish output. What remains to be seen is how the base packages can version truly independently since the meta package will define specific versions it depends on. When you depend on a version of AspNetCore.All you depend on the versions it defines. It seems that this might now require a new SDK release each time the underlying packages change so that you can work with a new version of the meta-package, while still having the packages locally.

Generally though, this new simplified file should make it very easy for people working outside of VS 2017 to get up and running with a hand coded project file if they so wish.

Another change you’ll notice is that the TargetFramework is now netcoreapp2.0 which defines that this project will run on .NET Core 2.0.

Simplified Program.cs

Next up, program CS got some refinement. To compare, in an ASP.NET Core 1.1 application, the class looks like this:

Now in 2.0 it looks like this:

This revised version utilises a C#6 syntax to build an expression body method called BuildWebHost. This method uses a new static method WebHost.CreateDefaultBuilder. This method provides a shortcut to get an IWebHostBuilder. Behind the scenes, this method will do a few things for us, saving a few lines of code. As per the source documentation, it will:

  1. Use Kestrel as the web server
  2. Set the ContentRootPath
  3. Load configuration from the default locations such as appsettings.json, user secrets and the environment variables. Note that configuration used to be defined inside the constructor of the Startup.cs file
  4. Configure the ILoggerFactory to log to console and debug. This also used to be inside Startup.cs, often defined in the Configure method
  5. Enables IISIntegration
  6. Add the developer exception page when in the development environment

If you want to look at the full source for the WebHost class, which lives inside the MetaPackages repository, you can take a look at the code up on GitHub.

Simplified Startup.cs

Startup.cs has also been simplified, mostly as a result of moving the logging and configuration code into the WebHost builder.

Previously a default ASP.NET Core 1.1 project would look like this:

Now it looks like this:

The constructor no longer takes an injected IHostingEnvironment, instead taking the IConfiguration dependency. With configuration sources already defined the constructor just sets the Configuration properly so we can access it when needed.

ConfigureServices is unchanged, but Configure is slightly shorter since it no longer has to define the logging.

Summary

There you have it. A quick tour of the basic structural changes you can expect when you start working with ASP.NET Core 2.0. I’m pretty happy with the changes. My initial concerns around the AspNetCore.All package are hopefully unfounded and the shifting of logging and configuration into extensions on IWebHostBuilder makes sense. The startup file is nice and clean by default and pretty much all code we add will be specifically to include services we require and to setup our pipeline.

ASP.NET Core Anatomy (Part 4) – Invoking the MVC Middleware Dissecting and understanding the internals of ASP.NET Core

In the first three parts of this series I looked at what happens when we call AddMvcCore, AddMvc and UseMvc as part of the application startup. Once the MVC services and middleware have been registered in our ASP.NET Core application, MVC is now available to handle HTTP requests.

In this post I want to cover the initial steps that happen when a request flows into the MVC middleware. This is quite a complex area to split up and write about. I’ve broken it out into what I consider a reasonable flow through the code, ignoring for now certain branches of behaviour and details to constrain this post to something that I hope is digestible in one sitting. Where I gloss over deeper implementation details I will highlight these cases and will write about those elements in future posts.

As with my earlier blog posts, I’ve used the original project.json version (1.1.2) of the MVC source code, since I’ve still not found a reliable way to debug through the source of MVC while including source from the other components such as Routing.

So, let’s begin and look at how MVC matches a request through to an available route and finally to an action which can handle that request. As a quick refresher, the middleware pipeline, configured in Startup.cs file of an ASP.NET Core application defines the flow of the request handling. Each middleware component will be invoked in order, until one of the middleware components determines it can provide a suitable response.

I’m using the MvcSandbox, included with the MVC solution to work with MVC while I write this series. The Configure method looks like this:

Assuming none of the earlier middleware handles the request, we reach the MVC pipeline and middleware, included in our pipeline with the UseMvc call. Once the request reaches the MVC pipeline, the middleware we hit is the RouterMiddleware. Its Invoke method looks like this:

The first thing that Invoke does is construct a new RouteContext passing in the current HttpContext object into the constructor.

The HttpContext is used to set a parameter on the RouteContext class and then a new RouteData object is instantiated and set.

Back inside Invoke; the injected IRouter, in this case the RouteCollection that was created during the UseMvc setup, is added to a List of IRouter objects on the RouteContext.RouteData object. Something worth highlighting is that the RouteData object uses a late initialisation pattern for its collections, only allocating them if they are called. This pattern shows the consideration that has to be made for performance inside a large framework such as ASP.NET Core.

For example, here is how the Routers property is defined:

The first time this property is accessed, a new List will be allocated and stored in the backing field.

Back in Invoke; RouteAsync is called on the RouteCollection.

The first thing the RouteAsync implementation does on the RouteCollection is to create a RouteDataSnapshot. As the comment indicates, allocating a RouteData object is not something that should happen for every route being processed. To avoid that, the snapshot of the RouteData object is created once and this allows it to be reset for each iteration. This is another example of where the ASP.NET Core team have coded with performance in mind.

The snapshot is achieved by calling PushState on the RouteData class.

The first thing it does is create a List<IRouter>. To be as performant as possible it only allocates a list if the private field containing the RouteData routers has at least one IRouter in it. If so, a new list is created, with the correct size to avoid internal Array.CopyTo calls occurring to resize the Lists underlying array. Essentially this method now has a copied instance of the RouteData’s internal IRouter list.

Next a RouteDataSnapshot object is created. RouteDataSnapshot is defined as a struct. Its constructor signature looks like this:

The RouteCollection calls PushState with null values for all of the parameters. In cases where the PushState method be called with a non-null IRoute parameter then it gets added to the Routers list. Values and DataTokens are processed in the same way. If any are included on the PushState parameters, the appropriate items in the Values and DataTokens properties on RouteData are updated.

Finally, the snapshot is returned to RouteAsync inside the RouteCollection.

Next a for loop is started which will loop up to the Count property value. Count simply exposes the Count of the Routes (List<IRouter>) on the RouteCollection.

Inside the loop it first gets a route using the value of the loop. This calls to

So this simply returns an IRouter from the List for the required index. In the MvcSandbox example, the first IRouter in the list at index 0 is the AttributeRoute.

Inside a Try/Finally block, the RouteAsync method is called on the IRouter (the AttributeRoute). Ultimately what’s happening here is we’re hoping to find a suitable Handler (a RequestDelegate) that matches the route data.

We’ll look into more detail about what happens inside the AttributeRoute.RouteAsync method in a future post as there’s a lot happening there and at the moment we’ve not defined any AttributeRoutes within the MvcSandbox. In our case, because there are no AttributeRoutes defined, the Handler value remains null.

Inside the finally block, when the Handler is null, the Restore method is called on the RouteDataSnapshot. This method will restore the RouteData object back to its state when the snapshot was created. Because the RouteAsync method may have modified RouteData during processing, this ensures that we are back to an initial state for the object.

Within the MvcSandbox example, the second IRouter in the list is the named “default” route which is an instance of Route. This class does not override the RouteAsync method on the base, so in this case, the code inside the RouteBase abstract class is called.

First it calls a private method, EnsureMatcher which looks like this:

This method will instantiate a new TemplateMatcher, passing in two parameters. Again, this appears to be an allocation optimisation to ensure that this object is only created at the point it is needed, after the properties which are passed to its constructor are available.

In case you’re wondering where those properties are set; that happens inside the constructor of the RouteBase class. This constructor was called when the default Route was first created as a result of calling MapRoute inside the UseMvc extension, called by the Configure method of the MvcSandbox Startup class.

Inside the RouteBase.RouteAsync method, the next call is to EnsureLoggers which looks like this:

This method gets the ILoggerFactory instance from the RequestServices and uses it to setup two ILoggers. The first is for the RouteBase class itself and the second will be used by the RouteConstraintMatcher.

Next it stores a local variable holding the path of the request being made, retrieved from the HttpContext.

Next, TryMatch on the TemplateMatcher is called, passing in the path of the request, along with any route data values. We’ll dive into the internals of TemplateMatcher in another post. For now, assume that TryMatch returns true, which is the case in our example. In cases where a match is not made a TaskCache.CompletedTask is returned. This is just a Task that is already set as completed.

Next, if there are any DataTokens (RouteValueDictionary objects) set, then the values on the context.RouteData.DataTokens are updated as necessary. As the comment suggests, this is only done if there are actually any values to be updated. The DataTokens property on RouteData is only created when its first called (lazily instantiated). Therefore, calling it when there are no values to update would risk allocating a new RouteValueDictionary when it may not be needed.

In our demo using MvcSandbox, there are no DataTokens so MergeValues is not called. However, for completeness, here is its code:

When called it loops any values in the RouteValueDictionary from the DataTokens parameter on the RouteBase class and updates the destination value for the matching key on the context.RouteData.DataTokens property.

Next, back in RouteAsync, RouteConstraintMatcher.Match is called. This static method loops over any IRouteContaints that are passed in and determines if they have all been met. Route constraints allow routes to be configured with additional matching rules. For example, a route parameter can be constrained to only match for integer values. In our case there are no constraints on our demo route so it returns true. We’ll look at a URL with constraints in another post.

A logger entry is made via an extension method on ILogger called MatchedRoute. This is an interesting pattern that makes it simple to reuse more complex formatting of logging messages for specific needs.

This class looks like this:

An action delegate is defined when the TreeRouterLoggerExtensions class is first constructed that defines how the logged message will be formatted.

When the MatchedRoute extension method is called, the route name and template string are passed as parameters. They are then passed to the _matchedRoute Action. This action creates a debug level log entry using the provided parameters. When debugging inside visual studio you will see this appear in the Output window.

Back inside RouteAsync; the OnRouteMatched method is called. This is defined as an abstract method on the RouteBase, so the implementation comes from the inheriting class. In our case it’s the Route class. Its OnRouteMatched override looks like this:

Its IRouter field called _target is added to the context.RouteData.Routers list. In this case it’s an MvcRouteHandler which is the default handler for MVC.

The RouteAsync method is then called on the MvcRouteHandler. The details of what this does are quite involved so I’ll save those to be the topic of a future post. In summary, MvcRouteHandler.RouteAsync will try to establish a suitable action method that should handle the request. One thing that is important to know for the context of this blog post is that when an action is found, the Handler property on the RouteContext is defined via a lambda expression. We can dive into that code another time but to summarise, a RequestDelegate is a function that accepts a HttpContext and can process a request.

Back inside the Invoke method on the RouterMiddleware, we might have a handler that MVC has determined can handle the requested route. If not then _logger.RequestDidNotMatchRoutes() is called. This is another example of the logger extension style we explored earlier. It will add a debug message, indicating that the route was not matched. In that case, the next RequestDelegate in the ASP.NET middleware pipeline is invoked because MVC has determined it cannot handle the request.

In common configurations for client web/api applications you will not have any middleware defined after UseMvc. In that case as we have reached the end of the pipeline a default 404 not found HTTP status code response is returned by ASP.NET Core.

In cases where we have a handler that can handle the route of the request, we flow through into the Invoke methods else block.

Here a new RoutingFeature is instantiated and added to the Features collection on the HttpContext. Briefly; features are a concept of ASP.NET Core that allows the server to define features that the request supports. This includes flowing of data through the lifetime of a request. Middleware such as the RouterMiddleware can add/modify the features collection and can use this as a mechanism to pass data through the request. In our case, the RouteData from the RouteContext is added as part of the IRoutingFeature definition, so that it can be used by other middleware and request handlers.

The method then invokes the Handler RequestDelegate which will ultimately process the request via the appropriate MVC action. At this point I’ll wrap up this post. What happens next, along with some of the other items I skipped over will form the next parts in this series.

Summary

We’ve seen how MVC is actually invoked as part of the middleware pipeline. Upon invocation, the MVC RouterMiddleware determines if MVC knows how to handle the incoming request path and values. If MVC has an action available that is intended to handle the route and route data on the request, then this handler is used to process the request and provide the response.

Other Posts In This Series

Migration

Migrating from project.json to csproj using Visual Studio 2017 Moving a real world ASP.NET Core application using VS2015 project.json to VS2017 and csproj

This past weekend I spent a few hours working on migrating the Humanitarian Toolbox allReady project over to the new csproj based Visual Studio 2017 solution format. I’ve written a few times about the plans to retire the project.json file introduced by ASP.NET Core. If you want some background you can read those posts first (don’t worry, I’ll wait)…

The summary is that in Visual Studio 2017 and the final ASP.NET Core SDK tooling, the project.json file was removed in favour of a revised csproj file. This has left some people confused and wondering why project.json is missing in Visual Studio 2017. The decision was mostly made to support customers reliant on using MsBuild and to support migrating larger projects to .NET Core.

Moving from project.json to csproj

In this post I wanted to cover the steps I took to migrate a real ASP.NET Core application to Visual Studio 2017. I’ve recorded the steps that I took and the issues I faced, along with any solutions I was able to find.

TL;DR; It wasn’t as painful as I’d expected it might be, but it also wasn’t entirely automated. There were a few issues along the way, which required some intervention on my part to finally get everything building correctly.

My first trial attempt at this process was actually about 2 weeks ago. At that time I was using the first RTM version of VS 2017 and I started by simply opening our allReady solution file with Visual Studio 2017. 

Visual Studio presents a dialog showing that any project.json projects will be migrated.

Visual Studio 2017 project.json migration dialog

After accepting that message VS will do it’s thing. Under the covers it’s calling the dotnet migrate command which handles the conversion for you. I decided to use Visual Studio 2017 for the migration, rather than the command line as I expect that’s the approach most people will be taking.

During migration; which took only a few seconds for me, you’ll see a progress window from Visual Studio.

ASP.NET Core project.json migration progress

Once the migration is completed you are shown a migration report.

project.json to csproj migration report

This initially led me to believe that things had gone pretty well. In fact, when I tried a restore and build things were not quite so positive. I was shown a staggering 982 errors and 3 warnings. It was late and I was tired, so I decided to abandon attempt one at that point! That was a problem for my future self!

Attempt Two

This weekend I was finally in a position to have another try at the migration. I took the latest version of our code and before starting, updated Visual Studio 2017 because I’d seen that it had some updates pending.

I repeated the initial steps as before and this time, although the result was still not looking great, it was a fair amount improved on attempt 1. Lesson 1 – Always do migrations with the most updated version of the IDE! This time I was at 199 errors and 2 warnings.

project.json Migration Errors

The errors were mostly relating to dependencies so it looked to be a restore issue. I tried cleaning the solution, restarting Visual Studio and even manually deleting the bin and obj folders. Each time I was hit with a wall of red errors when trying to build.

At this point I jumped onto Google and found a suggestion to clear the local nuget cache. I navigated to %USERPROFILE%/.nuget and deleted the contents.

Upon doing that, things were starting to look a little better. The errors cleared but I now had some warnings to deal with and the site would not run.

The second warning was a little strange.

The referenced project ‘..\..\Backup\Web-App\AllReady\AllReady.csproj’ does not exist.

The unit test project was referencing the web project but using the backup folder path. The backup folder is created during migration to hold the original project.json and xproj files in case you need to get back to a prior state. That folder of course didn’t include a csproj file. I removed the reference and re-added it, pointing it to the correct folder.

I also took the chance to move the backup folder so that it was outside of my main solution structure, just in case it was causing or masking any other issues.

Attempting to build at this stage reduced my errors but I still had 29 warnings. To be sure that Visual Studio 2017 wasn’t partly to blame, I also tried using the command line dotnet restore commands directly. However, I was getting warnings there also.

Inside Visual Studio the warning looked like this:

Detected package downgrade: Microsoft.Extensions.Configuration.Abstractions from 1.1.0 to 1.0.2

Trying to run the application at this stage resulted in the following exception:

System.IO.FileLoadException: ‘Could not load file or assembly ‘Microsoft.Extensions.Configuration.Abstractions, Version 1.1.0.0 … The located assembly’s manifest definition does not match the assembly reference.

For some reason the migration had chosen version 1.1.0 for the Microsoft.VisualStudio.Web.BrowserLink dependency when it created the csproj file.

<PackageReference Include="Microsoft.VisualStudio.Web.BrowserLink" Version="1.1.0" />

The solution was to change it to 1.0.1 since allReady is targeting the LTS stream of ASP.NET Core currently. All of the other dependencies were registered with their LTS versions. This rouge 1.1.x dependency was therefore causing version conflicts.

<PackageReference Include="Microsoft.VisualStudio.Web.BrowserLink" Version="1.0.1" />

At this point, building the solution was getting a bit further. I now had a single warning left for the web project. This warning stated that AddUserSecrets(IConfigurationBuilder) is now an obsolete method.

‘ConfigurationExtensions.AddUserSecrets(IConfigurationBuilder)’ is obselete. ‘This method is obsolete and will be removed in a future version. The recommended alternative is .AddUserSecrets(string userSercretsId) or .AddUserSecrets<TStartup>()..’

The warning explains that we were now expected to pass in the user secrets key as a string to the method or use .AddUserSecrets<TStartup> which is syntactic sugar over AddUserSecrets(this IConfigurationBuilder configuration, Assembly assembly). Previously the storage location for the user secrets ID was the project.json file. It’s now moved to an assembly attribute. The previous overload is not guaranteed to return the correct Assembly so this could produce issues. You can read a fuller explanation from the team on the GitHub repo.

In our case I changed the appropriate line in Startup.cs from

builder.AddUserSecrets();

to

builder.AddUserSecrets("aspnet5-AllReady-468aac76-4430-43e6-848e-f4a3b90d61d0");

Test Project Changes

At this point the web project was building but there were issues inside the test project:

Test project warning after project.json migration

Found conflicts between different versions of the same dependent assembly that could not be resolved. These reference conflicts are listed in the build log when log verbosity is set to detailed.

Some of the dependencies we had been using had newer versions so I took a guess that perhaps I needed to update the Xunit / test related ones. I changed these three dependencies:

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0-preview-20170106-08" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.2.0-beta5-build1225" />
<PackageReference Include="xunit" Version="2.2.0-beta5-build3474" />

to

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
<PackageReference Include="xunit" Version="2.2.0" />

Finally the entire solution was building, tests were running and I could start the website locally.

Initial thoughts on csproj and Visual Studio 2017

Generally, everything seems to work once inside Visual Studio 2017. The migration process was a little more effort than I would have liked though.

No auto complete

In Visual Studio 2015 we had version auto complete for the dependencies from NuGet when editing project.json. We’ve lost this in Visual Studio 2017 which is a shame. It now pretty much forces me into the NuGet Package Manager which I find slower. I understand that autocomplete might come back at some point, but it’s a shame we’re lacking it upon release. 

Less easy to scan

Personally, I still prefer the project.json format for readability. There’s still a little more noise in the XML than we had inside the JSON version. That said, the work done to the revised csproj format is great and it’s a vast improvement on the csproj of the past.

Summing Up

Overall the process wasn’t too harrowing. Visual Studio and dotnet migrate handled most of the work for me. It was a shame that I had to resort on a few occasions to Google for manual solutions to solve problems that I’m sure should have been addressed by the tooling. The restore of the dependencies not working without clearing the cache was an unexpected problem.

But with the work of migration behind me I’m ready to move on. I’ve reached the acceptance stage on my project.json grieving process!

Anyone new to ASP.NET Core, using it for the first time in Visual Studio 2017 will probably look at older blog posts and wonder what we were all moaning about. The csproj is a great improvement on what we had in the past, so if you’ve never seen project.json, there’s nothing to miss and only improvements to enjoy.

ASP.NET Core Anatomy (Part 3) – UseMvc Dissecting and understanding the internals of ASP.NET Core

In parts one and two of this series I looked at the two main IServiceCollection extensions (AddMvcCore and AddMvc) in ASP.NET Core. These are used to add in the required MVC services when you plan to use the MVC middleware in your application.

The next thing you are required to do to enable MVC in your ASP.NET Core application is to include the UseMvc IApplicationBuilder extension from the Configure method of your Startup class. This method registers the MVC middleware into your application pipeline so that the MVC framework can handle requests and return the appropriate response (usually a view result or some JSON). In this post I will take a look at what happens when the UseMvc method is called during the application startup.

As with the earlier parts of this series, I’m currently using the rel/1.1.2 codebase from the MVC repository for my investigations. I’m still using the slightly older project.json based code as at the moment there don’t seem to be simple ways to debug into multiple ASP.NET Core source code assemblies in VS2017.

UseMvc is an extension on the IApplicationBuilder which takes an Action delegate of IRouteBuilder. The IRouteBuilder will be used to configure the routing for MVC. There is a more basic variant of the UseMvc method which does not required any parameters. This method simply calls down into the main version, passing in an empty delegate like so…

The main UseMvc method looks like this…

Let’s dissect what this method does. First it does a check to see if the IServiceProvider has an MvcMarkerService service registered in it. To do this it uses the ApplicationServices property which exposes the current IServiceProvider. Calling GetService on the service provider will try to locate a registered implementation for the requested Type. The MvcMarkerService is registered when AddMvcCore executes and as a result, if it is not found, it indicates that either AddMvc or AddMvcCore was not called during ConfigureServices. Marker services like this are used in quite a few places to help calling code verify proper registration of fundamental dependencies before they try to execute. The MvcMarkerService is just an empty class with no actual code.

Next, UseMvc requests a MiddlewareFilterBuilder from the ServiceProvider and sets its ApplicationBuilder using the IApplicationBuilder.New() method. The ApplicationBuilder creates and returns a new instance copy of itself when this method is called.

Next, UseMvc initialises a new RouteBuilder, setting the default handler to the registered MvcRouteHandler. At this point, the DI magic really kicks in and a bunch of dependencies start getting instantiated. MvcRouteHandler is requested and because it has some dependencies in its constructor, various other classes get initialised. Each constructor for these classes can also require additional dependencies which then also get created. This is a great example of how the DI system works. While all of the interfaces and concrete implementations are registered in the container during ConfigureServices, the actual objects are only created at the point they are required to be injected into something. By new-ing up the RouteBuilder, this snowball of object creation begins to occur, until all of the required dependencies have been constructed. Some of these are registered as singletons and will then live for the lifetime of the application, being returned whenever they are requested from the ServiceProvider. Others, might be constructed for each request or in some cases, constructed uniquely every time they are required. Understanding the details of how dependency injection works and specifically the ASP.NET Core ServiceProvider is out of scope for this post.

After creating the RouteBuilder, the Action<IRouteBuilder> delegate action is called which takes the RouteBuilder as its only parameter. In the MvcSandbox sample that I’m using to debug and dissect MVC we call the UseMvc method, passing in a delegate function via a lambda. This function maps a single route named “default” as follows.

This calls the MapRoute method which is an extension on the IRouteBuilder. The main MapRoute method looks like this:

This requests an instance of an IInlineConstraintResolver which is resolved by the ServiceProvider to a DefaultInlineConstraintResolver. This class takes an IOptions<RouteOptions> in its constructor.

The Microsoft.Extensions.Options assembly, as part of its work, calls into the MvcCoreRouteOptionsSetup.Configure method which takes a RouteOptions parameter. This adds a new constraint map of type KnownRouteValueConstraint to the Dictionary of constraint maps. This Dictionary is initialised with a number of default constraints when the RouteOptions is first constructed. I’ll look into the routing code in a little more detail in a future post.

A new Route object is constructed using the name and template provided. In our case this is added the RouteBuilder.Routes List. By this point, running the sample MvcSandbox application, our route builder now has a single route added.

Note that the MvcApplicationBuilderExtensions class also includes an extension method called UseMvcWithDefaultRoute. This method includes a call to UseMvc, passing in a hardcoded default route.

This route is defined with the same name and template as defined in the MvcSandbox application. Therefore it could have been used as a slight code saver within the MvcSandbox Startup class. For the most basic of MVC applications it’s possible that using this helper extension method might be enough for your application routing requirements. In most real-world cases I suspect you’ll end up passing in a more complete customised set of routes. It’s a nice shorthand though if you want to start with the basic routing template. It’s also possible to call UseMvc() without a parameter and instead use the attribute routing approach. 

Next, the static AttributeRouting.CreateAttributeMegaRoute method is called and the resulting route is added to the Routes List (at index zero) on the RouteBuilder. CreateAttributeMegaRoute is documented with the summary “Creates an attribute route using the provided services and provided target router.” and it looks like this:

This method, creates a new AttributeRoute which implements the IRouter interface. Its constructor looks like this:

In its constructor it takes an IActionDescriptorCollectionProvider, an IServiceProvider and a Func which takes an array of ActionDescriptor’s and which returns an IRouter. By default, with the AddMvc registered services, it will get an instance of ActionDescriptorCollectionProvider which is a singleton object registered with the ServiceProvider. ActionDescriptors represent the available MVC actions that have been created and discovered within the application. We’ll look at those another time.

The code (inside the CreateAttributeMegaRoute method) which creates the new AttributeRoute uses a lambda to define the code for the Func<ActionDescriptor[], IRouter>. In this case the delegate function requests an MvcAttributeRouteHandler from the ServiceProvider. Because this is registered as transient, the ServiceProvider will return a new instance of an MvcAttributeRouteHandler each time one is requested. The delegate code then sets the Actions property (an array of ActionDescriptions) on the MvcAttributeRouteHandler using the incoming ActionDescriptions array and finally the new handler is returned.

Back to UseMvc; it finishes by calling the UseRouter extension on the IApplicationBuilder. The object passed into UseRouter is created by calling Build on the RouteBuilder. This method will add the routes into the RouteCollection. Internally the RouteCollection keeps track of which routes are named and which are unnamed. In our MvcSandbox example we end up with a named “default” route and an unnamed AttributeRoute.

UseRouter has two signatures. In this case we’re passing in a IRouter so the following method is called:

This does a check for a RoutingMarkerService within the ServiceProvider. Assuming this is registered as expected  it adds the RouterMiddleware into the middeware pipeline. It is this middleware which will use the IRouter to try and match up requests to the controller and action within MVC which should handle them. The details of this process will feature in a future blog post.

At this point the application pipeline is configured and the application is ready to start receiving requests. That’s a good point to stop for this post.

Summary

In this post we’ve looked at UseMvc and seen that it sets up the MiddlewareFilterBuilder which will be used later. It then also gets an IRouter via a RouteBuilder and much of the work at this stage is around registering routes. Once everything is setup, the routing middleware is registered into the pipeline. It’s this code that knows how to inspect incoming requests and map the paths to the appropriate controllers and actions that can handle them.

Other Posts In This Series