Getting started with ASP.NET Core 2.0 Preview 1 A quick tour of the main structural changes

At Microsoft Build 2017 yesterday, the formal announcement of ASP.NET Core 2.0 and .NET Core 2.0 was made.

ASP.NET Core 2.0 preview announcement

.NET Core 2.0 preview announcement

For me it was no surprise that we were close with these announcements as I keep an eye on the ASP.NET Core repositories on GitHub where there have been preview 1 branches for a little while. Additionally, the team have shared some information on the weekly ASP.NET Community Standup shows that have helped us understand some of the changes coming our way.

In this post I want to focus in on a few changes to the basic structure of ASP.NET Core 2.0 applications which simplify the code needed to get started. Before looking at those elements, I felt it would be worth sharing the steps I took to get started with ASP.NET Core 2.0 Preview 1 on my development machine.

Getting started with ASP.NET Core 2.0 preview 1

The first step was to get the preview SDK for .NET Core 2.0 which can be safely installed alongside any prior 1.x SDK versions. The announcement post provides a link to download the SDK.

The next step was to install the SDK. Nice and easy!

Installing .NET Core 2.0 SDK

Initially I tried creating a project in the existing Visual Studio 2017 IDE. I did so by starting with an ASP.NET Core 1.1 project and simply updating it to target netcoreapp2.0 and the latest ASP.NET Core 2.0 packages. However, this was problematic since I couldn’t restore or build the solution inside Visual Studio. I did manage to do both of those tasks at the command line level using dotnet restore and dotnet build. Finally, running dotnet run I was up and running with a small sample API site. 

After querying the lack of Visual Studio support on Twitter, Damian Edwards confirmed that I would need to use the preview version of Visual Studio 2017 in order to work with ASP.NET 2.0 preview projects. So I went off to download and then install it. According to their website this is safe to install side-by-side with other versions. If you want to follow along, download Visual Studio 2017 preview from here.

Installing Visual Studio 2017 preview

Now that I had VS 2017 preview installed on my machine I decided to start fresh and try creating a new project directly inside the IDE. The new project dialog for an ASP.NET Core web project has been updated so I could simply choose to target that from the UI.

ASP.NET Core 2.0 project template

After a few moments I was up and running, working on a new ASP.NET 2.0 web application. A very painless experience so well done and thanks to the team for the work they’ve done to get us started on this preview release.

A Quick Tour of ASP.NET Core 2.0 preview 1

Now that I have a default ASP.NET Core 2.0 project on my machine, I want to highlight a few of the initial changes that you’ll see.

Simplified CSPROJ

After the move back to csproj that came with Visual Studio 2017, the new structure for these project files has been a great improvement on the past. Gone are the ugly GUIDs and the requirement to specify all files. With this release, the team have simplified things a step further.

In the mainstream Visual Studio 2017 release, targeting ASP.NET Core 1.1 a default csproj file looks like this:

In the latest version, we now have a single meta-package we reference to bring in all of the ASP.NET Core packages. This reduces a default csproj to just these few lines:

At first this concerned me a little. A key message behind the original ASP.NET Core release, was a “pay to play” model. You could specify only the packages you actually needed in your application, no longer bringing a dependency on a whole monolith framework. You could bring each version of each component that you actually used. This also resulted in a smaller footprint for your published application. My first impression was that we were taking a bit of a step back by instead referencing a single master package.

However after reading the announcement and chatting on Twitter, the consensus seems to be that this is not a problem. The SDK ships with the packages included with it, so you are never pulling these from NuGet directly. That had been a concern for me since at work we build our projects inside Docker containers. If each new build needed to pull all of the packages, even those we didn’t need, across the Internet, it could slow down our builds.

However, since they are local with the SDK, this is not a problem. Then I had a concern about a large deployment when you publish your application, including binaries that my application doesn’t even use. This also seems to be solved, as Microsoft now have a package trimming feature that they explain will exclude the binaries you are not using for your publish output. What remains to be seen is how the base packages can version truly independently since the meta package will define specific versions it depends on. When you depend on a version of AspNetCore.All you depend on the versions it defines. It seems that this might now require a new SDK release each time the underlying packages change so that you can work with a new version of the meta-package, while still having the packages locally.

Generally though, this new simplified file should make it very easy for people working outside of VS 2017 to get up and running with a hand coded project file if they so wish.

Another change you’ll notice is that the TargetFramework is now netcoreapp2.0 which defines that this project will run on .NET Core 2.0.

Simplified Program.cs

Next up, program CS got some refinement. To compare, in an ASP.NET Core 1.1 application, the class looks like this:

Now in 2.0 it looks like this:

This revised version utilises a C#6 syntax to build an expression body method called BuildWebHost. This method uses a new static method WebHost.CreateDefaultBuilder. This method provides a shortcut to get an IWebHostBuilder. Behind the scenes, this method will do a few things for us, saving a few lines of code. As per the source documentation, it will:

  1. Use Kestrel as the web server
  2. Set the ContentRootPath
  3. Load configuration from the default locations such as appsettings.json, user secrets and the environment variables. Note that configuration used to be defined inside the constructor of the Startup.cs file
  4. Configure the ILoggerFactory to log to console and debug. This also used to be inside Startup.cs, often defined in the Configure method
  5. Enables IISIntegration
  6. Add the developer exception page when in the development environment

If you want to look at the full source for the WebHost class, which lives inside the MetaPackages repository, you can take a look at the code up on GitHub.

Simplified Startup.cs

Startup.cs has also been simplified, mostly as a result of moving the logging and configuration code into the WebHost builder.

Previously a default ASP.NET Core 1.1 project would look like this:

Now it looks like this:

The constructor no longer takes an injected IHostingEnvironment, instead taking the IConfiguration dependency. With configuration sources already defined the constructor just sets the Configuration properly so we can access it when needed.

ConfigureServices is unchanged, but Configure is slightly shorter since it no longer has to define the logging.

Summary

There you have it. A quick tour of the basic structural changes you can expect when you start working with ASP.NET Core 2.0. I’m pretty happy with the changes. My initial concerns around the AspNetCore.All package are hopefully unfounded and the shifting of logging and configuration into extensions on IWebHostBuilder makes sense. The startup file is nice and clean by default and pretty much all code we add will be specifically to include services we require and to setup our pipeline.

ASP.NET Core Anatomy (Part 4) – Invoking the MVC Middleware Dissecting and understanding the internals of ASP.NET Core

In the first three parts of this series I looked at what happens when we call AddMvcCore, AddMvc and UseMvc as part of the application startup. Once the MVC services and middleware have been registered in our ASP.NET Core application, MVC is now available to handle HTTP requests.

In this post I want to cover the initial steps that happen when a request flows into the MVC middleware. This is quite a complex area to split up and write about. I’ve broken it out into what I consider a reasonable flow through the code, ignoring for now certain branches of behaviour and details to constrain this post to something that I hope is digestible in one sitting. Where I gloss over deeper implementation details I will highlight these cases and will write about those elements in future posts.

As with my earlier blog posts, I’ve used the original project.json version (1.1.2) of the MVC source code, since I’ve still not found a reliable way to debug through the source of MVC while including source from the other components such as Routing.

So, let’s begin and look at how MVC matches a request through to an available route and finally to an action which can handle that request. As a quick refresher, the middleware pipeline, configured in Startup.cs file of an ASP.NET Core application defines the flow of the request handling. Each middleware component will be invoked in order, until one of the middleware components determines it can provide a suitable response.

I’m using the MvcSandbox, included with the MVC solution to work with MVC while I write this series. The Configure method looks like this:

Assuming none of the earlier middleware handles the request, we reach the MVC pipeline and middleware, included in our pipeline with the UseMvc call. Once the request reaches the MVC pipeline, the middleware we hit is the RouterMiddleware. Its Invoke method looks like this:

The first thing that Invoke does is construct a new RouteContext passing in the current HttpContext object into the constructor.

The HttpContext is used to set a parameter on the RouteContext class and then a new RouteData object is instantiated and set.

Back inside Invoke; the injected IRouter, in this case the RouteCollection that was created during the UseMvc setup, is added to a List of IRouter objects on the RouteContext.RouteData object. Something worth highlighting is that the RouteData object uses a late initialisation pattern for its collections, only allocating them if they are called. This pattern shows the consideration that has to be made for performance inside a large framework such as ASP.NET Core.

For example, here is how the Routers property is defined:

The first time this property is accessed, a new List will be allocated and stored in the backing field.

Back in Invoke; RouteAsync is called on the RouteCollection.

The first thing the RouteAsync implementation does on the RouteCollection is to create a RouteDataSnapshot. As the comment indicates, allocating a RouteData object is not something that should happen for every route being processed. To avoid that, the snapshot of the RouteData object is created once and this allows it to be reset for each iteration. This is another example of where the ASP.NET Core team have coded with performance in mind.

The snapshot is achieved by calling PushState on the RouteData class.

The first thing it does is create a List<IRouter>. To be as performant as possible it only allocates a list if the private field containing the RouteData routers has at least one IRouter in it. If so, a new list is created, with the correct size to avoid internal Array.CopyTo calls occurring to resize the Lists underlying array. Essentially this method now has a copied instance of the RouteData’s internal IRouter list.

Next a RouteDataSnapshot object is created. RouteDataSnapshot is defined as a struct. Its constructor signature looks like this:

The RouteCollection calls PushState with null values for all of the parameters. In cases where the PushState method be called with a non-null IRoute parameter then it gets added to the Routers list. Values and DataTokens are processed in the same way. If any are included on the PushState parameters, the appropriate items in the Values and DataTokens properties on RouteData are updated.

Finally, the snapshot is returned to RouteAsync inside the RouteCollection.

Next a for loop is started which will loop up to the Count property value. Count simply exposes the Count of the Routes (List<IRouter>) on the RouteCollection.

Inside the loop it first gets a route using the value of the loop. This calls to

So this simply returns an IRouter from the List for the required index. In the MvcSandbox example, the first IRouter in the list at index 0 is the AttributeRoute.

Inside a Try/Finally block, the RouteAsync method is called on the IRouter (the AttributeRoute). Ultimately what’s happening here is we’re hoping to find a suitable Handler (a RequestDelegate) that matches the route data.

We’ll look into more detail about what happens inside the AttributeRoute.RouteAsync method in a future post as there’s a lot happening there and at the moment we’ve not defined any AttributeRoutes within the MvcSandbox. In our case, because there are no AttributeRoutes defined, the Handler value remains null.

Inside the finally block, when the Handler is null, the Restore method is called on the RouteDataSnapshot. This method will restore the RouteData object back to its state when the snapshot was created. Because the RouteAsync method may have modified RouteData during processing, this ensures that we are back to an initial state for the object.

Within the MvcSandbox example, the second IRouter in the list is the named “default” route which is an instance of Route. This class does not override the RouteAsync method on the base, so in this case, the code inside the RouteBase abstract class is called.

First it calls a private method, EnsureMatcher which looks like this:

This method will instantiate a new TemplateMatcher, passing in two parameters. Again, this appears to be an allocation optimisation to ensure that this object is only created at the point it is needed, after the properties which are passed to its constructor are available.

In case you’re wondering where those properties are set; that happens inside the constructor of the RouteBase class. This constructor was called when the default Route was first created as a result of calling MapRoute inside the UseMvc extension, called by the Configure method of the MvcSandbox Startup class.

Inside the RouteBase.RouteAsync method, the next call is to EnsureLoggers which looks like this:

This method gets the ILoggerFactory instance from the RequestServices and uses it to setup two ILoggers. The first is for the RouteBase class itself and the second will be used by the RouteConstraintMatcher.

Next it stores a local variable holding the path of the request being made, retrieved from the HttpContext.

Next, TryMatch on the TemplateMatcher is called, passing in the path of the request, along with any route data values. We’ll dive into the internals of TemplateMatcher in another post. For now, assume that TryMatch returns true, which is the case in our example. In cases where a match is not made a TaskCache.CompletedTask is returned. This is just a Task that is already set as completed.

Next, if there are any DataTokens (RouteValueDictionary objects) set, then the values on the context.RouteData.DataTokens are updated as necessary. As the comment suggests, this is only done if there are actually any values to be updated. The DataTokens property on RouteData is only created when its first called (lazily instantiated). Therefore, calling it when there are no values to update would risk allocating a new RouteValueDictionary when it may not be needed.

In our demo using MvcSandbox, there are no DataTokens so MergeValues is not called. However, for completeness, here is its code:

When called it loops any values in the RouteValueDictionary from the DataTokens parameter on the RouteBase class and updates the destination value for the matching key on the context.RouteData.DataTokens property.

Next, back in RouteAsync, RouteConstraintMatcher.Match is called. This static method loops over any IRouteContaints that are passed in and determines if they have all been met. Route constraints allow routes to be configured with additional matching rules. For example, a route parameter can be constrained to only match for integer values. In our case there are no constraints on our demo route so it returns true. We’ll look at a URL with constraints in another post.

A logger entry is made via an extension method on ILogger called MatchedRoute. This is an interesting pattern that makes it simple to reuse more complex formatting of logging messages for specific needs.

This class looks like this:

An action delegate is defined when the TreeRouterLoggerExtensions class is first constructed that defines how the logged message will be formatted.

When the MatchedRoute extension method is called, the route name and template string are passed as parameters. They are then passed to the _matchedRoute Action. This action creates a debug level log entry using the provided parameters. When debugging inside visual studio you will see this appear in the Output window.

Back inside RouteAsync; the OnRouteMatched method is called. This is defined as an abstract method on the RouteBase, so the implementation comes from the inheriting class. In our case it’s the Route class. Its OnRouteMatched override looks like this:

Its IRouter field called _target is added to the context.RouteData.Routers list. In this case it’s an MvcRouteHandler which is the default handler for MVC.

The RouteAsync method is then called on the MvcRouteHandler. The details of what this does are quite involved so I’ll save those to be the topic of a future post. In summary, MvcRouteHandler.RouteAsync will try to establish a suitable action method that should handle the request. One thing that is important to know for the context of this blog post is that when an action is found, the Handler property on the RouteContext is defined via a lambda expression. We can dive into that code another time but to summarise, a RequestDelegate is a function that accepts a HttpContext and can process a request.

Back inside the Invoke method on the RouterMiddleware, we might have a handler that MVC has determined can handle the requested route. If not then _logger.RequestDidNotMatchRoutes() is called. This is another example of the logger extension style we explored earlier. It will add a debug message, indicating that the route was not matched. In that case, the next RequestDelegate in the ASP.NET middleware pipeline is invoked because MVC has determined it cannot handle the request.

In common configurations for client web/api applications you will not have any middleware defined after UseMvc. In that case as we have reached the end of the pipeline a default 404 not found HTTP status code response is returned by ASP.NET Core.

In cases where we have a handler that can handle the route of the request, we flow through into the Invoke methods else block.

Here a new RoutingFeature is instantiated and added to the Features collection on the HttpContext. Briefly; features are a concept of ASP.NET Core that allows the server to define features that the request supports. This includes flowing of data through the lifetime of a request. Middleware such as the RouterMiddleware can add/modify the features collection and can use this as a mechanism to pass data through the request. In our case, the RouteData from the RouteContext is added as part of the IRoutingFeature definition, so that it can be used by other middleware and request handlers.

The method then invokes the Handler RequestDelegate which will ultimately process the request via the appropriate MVC action. At this point I’ll wrap up this post. What happens next, along with some of the other items I skipped over will form the next parts in this series.

Summary

We’ve seen how MVC is actually invoked as part of the middleware pipeline. Upon invocation, the MVC RouterMiddleware determines if MVC knows how to handle the incoming request path and values. If MVC has an action available that is intended to handle the route and route data on the request, then this handler is used to process the request and provide the response.

Other Posts In This Series

Migration

Migrating from project.json to csproj using Visual Studio 2017 Moving a real world ASP.NET Core application using VS2015 project.json to VS2017 and csproj

This past weekend I spent a few hours working on migrating the Humanitarian Toolbox allReady project over to the new csproj based Visual Studio 2017 solution format. I’ve written a few times about the plans to retire the project.json file introduced by ASP.NET Core. If you want some background you can read those posts first (don’t worry, I’ll wait)…

The summary is that in Visual Studio 2017 and the final ASP.NET Core SDK tooling, the project.json file was removed in favour of a revised csproj file. This has left some people confused and wondering why project.json is missing in Visual Studio 2017. The decision was mostly made to support customers reliant on using MsBuild and to support migrating larger projects to .NET Core.

Moving from project.json to csproj

In this post I wanted to cover the steps I took to migrate a real ASP.NET Core application to Visual Studio 2017. I’ve recorded the steps that I took and the issues I faced, along with any solutions I was able to find.

TL;DR; It wasn’t as painful as I’d expected it might be, but it also wasn’t entirely automated. There were a few issues along the way, which required some intervention on my part to finally get everything building correctly.

My first trial attempt at this process was actually about 2 weeks ago. At that time I was using the first RTM version of VS 2017 and I started by simply opening our allReady solution file with Visual Studio 2017. 

Visual Studio presents a dialog showing that any project.json projects will be migrated.

Visual Studio 2017 project.json migration dialog

After accepting that message VS will do it’s thing. Under the covers it’s calling the dotnet migrate command which handles the conversion for you. I decided to use Visual Studio 2017 for the migration, rather than the command line as I expect that’s the approach most people will be taking.

During migration; which took only a few seconds for me, you’ll see a progress window from Visual Studio.

ASP.NET Core project.json migration progress

Once the migration is completed you are shown a migration report.

project.json to csproj migration report

This initially led me to believe that things had gone pretty well. In fact, when I tried a restore and build things were not quite so positive. I was shown a staggering 982 errors and 3 warnings. It was late and I was tired, so I decided to abandon attempt one at that point! That was a problem for my future self!

Attempt Two

This weekend I was finally in a position to have another try at the migration. I took the latest version of our code and before starting, updated Visual Studio 2017 because I’d seen that it had some updates pending.

I repeated the initial steps as before and this time, although the result was still not looking great, it was a fair amount improved on attempt 1. Lesson 1 – Always do migrations with the most updated version of the IDE! This time I was at 199 errors and 2 warnings.

project.json Migration Errors

The errors were mostly relating to dependencies so it looked to be a restore issue. I tried cleaning the solution, restarting Visual Studio and even manually deleting the bin and obj folders. Each time I was hit with a wall of red errors when trying to build.

At this point I jumped onto Google and found a suggestion to clear the local nuget cache. I navigated to %USERPROFILE%/.nuget and deleted the contents.

Upon doing that, things were starting to look a little better. The errors cleared but I now had some warnings to deal with and the site would not run.

The second warning was a little strange.

The referenced project ‘..\..\Backup\Web-App\AllReady\AllReady.csproj’ does not exist.

The unit test project was referencing the web project but using the backup folder path. The backup folder is created during migration to hold the original project.json and xproj files in case you need to get back to a prior state. That folder of course didn’t include a csproj file. I removed the reference and re-added it, pointing it to the correct folder.

I also took the chance to move the backup folder so that it was outside of my main solution structure, just in case it was causing or masking any other issues.

Attempting to build at this stage reduced my errors but I still had 29 warnings. To be sure that Visual Studio 2017 wasn’t partly to blame, I also tried using the command line dotnet restore commands directly. However, I was getting warnings there also.

Inside Visual Studio the warning looked like this:

Detected package downgrade: Microsoft.Extensions.Configuration.Abstractions from 1.1.0 to 1.0.2

Trying to run the application at this stage resulted in the following exception:

System.IO.FileLoadException: ‘Could not load file or assembly ‘Microsoft.Extensions.Configuration.Abstractions, Version 1.1.0.0 … The located assembly’s manifest definition does not match the assembly reference.

For some reason the migration had chosen version 1.1.0 for the Microsoft.VisualStudio.Web.BrowserLink dependency when it created the csproj file.

<PackageReference Include="Microsoft.VisualStudio.Web.BrowserLink" Version="1.1.0" />

The solution was to change it to 1.0.1 since allReady is targeting the LTS stream of ASP.NET Core currently. All of the other dependencies were registered with their LTS versions. This rouge 1.1.x dependency was therefore causing version conflicts.

<PackageReference Include="Microsoft.VisualStudio.Web.BrowserLink" Version="1.0.1" />

At this point, building the solution was getting a bit further. I now had a single warning left for the web project. This warning stated that AddUserSecrets(IConfigurationBuilder) is now an obsolete method.

‘ConfigurationExtensions.AddUserSecrets(IConfigurationBuilder)’ is obselete. ‘This method is obsolete and will be removed in a future version. The recommended alternative is .AddUserSecrets(string userSercretsId) or .AddUserSecrets<TStartup>()..’

The warning explains that we were now expected to pass in the user secrets key as a string to the method or use .AddUserSecrets<TStartup> which is syntactic sugar over AddUserSecrets(this IConfigurationBuilder configuration, Assembly assembly). Previously the storage location for the user secrets ID was the project.json file. It’s now moved to an assembly attribute. The previous overload is not guaranteed to return the correct Assembly so this could produce issues. You can read a fuller explanation from the team on the GitHub repo.

In our case I changed the appropriate line in Startup.cs from

builder.AddUserSecrets();

to

builder.AddUserSecrets("aspnet5-AllReady-468aac76-4430-43e6-848e-f4a3b90d61d0");

Test Project Changes

At this point the web project was building but there were issues inside the test project:

Test project warning after project.json migration

Found conflicts between different versions of the same dependent assembly that could not be resolved. These reference conflicts are listed in the build log when log verbosity is set to detailed.

Some of the dependencies we had been using had newer versions so I took a guess that perhaps I needed to update the Xunit / test related ones. I changed these three dependencies:

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0-preview-20170106-08" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.2.0-beta5-build1225" />
<PackageReference Include="xunit" Version="2.2.0-beta5-build3474" />

to

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
<PackageReference Include="xunit" Version="2.2.0" />

Finally the entire solution was building, tests were running and I could start the website locally.

Initial thoughts on csproj and Visual Studio 2017

Generally, everything seems to work once inside Visual Studio 2017. The migration process was a little more effort than I would have liked though.

No auto complete

In Visual Studio 2015 we had version auto complete for the dependencies from NuGet when editing project.json. We’ve lost this in Visual Studio 2017 which is a shame. It now pretty much forces me into the NuGet Package Manager which I find slower. I understand that autocomplete might come back at some point, but it’s a shame we’re lacking it upon release. 

Less easy to scan

Personally, I still prefer the project.json format for readability. There’s still a little more noise in the XML than we had inside the JSON version. That said, the work done to the revised csproj format is great and it’s a vast improvement on the csproj of the past.

Summing Up

Overall the process wasn’t too harrowing. Visual Studio and dotnet migrate handled most of the work for me. It was a shame that I had to resort on a few occasions to Google for manual solutions to solve problems that I’m sure should have been addressed by the tooling. The restore of the dependencies not working without clearing the cache was an unexpected problem.

But with the work of migration behind me I’m ready to move on. I’ve reached the acceptance stage on my project.json grieving process!

Anyone new to ASP.NET Core, using it for the first time in Visual Studio 2017 will probably look at older blog posts and wonder what we were all moaning about. The csproj is a great improvement on what we had in the past, so if you’ve never seen project.json, there’s nothing to miss and only improvements to enjoy.