Migration

Migrating from project.json to csproj using Visual Studio 2017 Moving a real world ASP.NET Core application using VS2015 project.json to VS2017 and csproj

This past weekend I spent a few hours working on migrating the Humanitarian Toolbox allReady project over to the new csproj based Visual Studio 2017 solution format. I’ve written a few times about the plans to retire the project.json file introduced by ASP.NET Core. If you want some background you can read those posts first (don’t worry, I’ll wait)…

The summary is that in Visual Studio 2017 and the final ASP.NET Core SDK tooling, the project.json file was removed in favour of a revised csproj file. This has left some people confused and wondering why project.json is missing in Visual Studio 2017. The decision was mostly made to support customers reliant on using MsBuild and to support migrating larger projects to .NET Core.

Moving from project.json to csproj

In this post I wanted to cover the steps I took to migrate a real ASP.NET Core application to Visual Studio 2017. I’ve recorded the steps that I took and the issues I faced, along with any solutions I was able to find.

TL;DR; It wasn’t as painful as I’d expected it might be, but it also wasn’t entirely automated. There were a few issues along the way, which required some intervention on my part to finally get everything building correctly.

My first trial attempt at this process was actually about 2 weeks ago. At that time I was using the first RTM version of VS 2017 and I started by simply opening our allReady solution file with Visual Studio 2017. 

Visual Studio presents a dialog showing that any project.json projects will be migrated.

Visual Studio 2017 project.json migration dialog

After accepting that message VS will do it’s thing. Under the covers it’s calling the dotnet migrate command which handles the conversion for you. I decided to use Visual Studio 2017 for the migration, rather than the command line as I expect that’s the approach most people will be taking.

During migration; which took only a few seconds for me, you’ll see a progress window from Visual Studio.

ASP.NET Core project.json migration progress

Once the migration is completed you are shown a migration report.

project.json to csproj migration report

This initially led me to believe that things had gone pretty well. In fact, when I tried a restore and build things were not quite so positive. I was shown a staggering 982 errors and 3 warnings. It was late and I was tired, so I decided to abandon attempt one at that point! That was a problem for my future self!

Attempt Two

This weekend I was finally in a position to have another try at the migration. I took the latest version of our code and before starting, updated Visual Studio 2017 because I’d seen that it had some updates pending.

I repeated the initial steps as before and this time, although the result was still not looking great, it was a fair amount improved on attempt 1. Lesson 1 – Always do migrations with the most updated version of the IDE! This time I was at 199 errors and 2 warnings.

project.json Migration Errors

The errors were mostly relating to dependencies so it looked to be a restore issue. I tried cleaning the solution, restarting Visual Studio and even manually deleting the bin and obj folders. Each time I was hit with a wall of red errors when trying to build.

At this point I jumped onto Google and found a suggestion to clear the local nuget cache. I navigated to %USERPROFILE%/.nuget and deleted the contents.

Upon doing that, things were starting to look a little better. The errors cleared but I now had some warnings to deal with and the site would not run.

The second warning was a little strange.

The referenced project ‘..\..\Backup\Web-App\AllReady\AllReady.csproj’ does not exist.

The unit test project was referencing the web project but using the backup folder path. The backup folder is created during migration to hold the original project.json and xproj files in case you need to get back to a prior state. That folder of course didn’t include a csproj file. I removed the reference and re-added it, pointing it to the correct folder.

I also took the chance to move the backup folder so that it was outside of my main solution structure, just in case it was causing or masking any other issues.

Attempting to build at this stage reduced my errors but I still had 29 warnings. To be sure that Visual Studio 2017 wasn’t partly to blame, I also tried using the command line dotnet restore commands directly. However, I was getting warnings there also.

Inside Visual Studio the warning looked like this:

Detected package downgrade: Microsoft.Extensions.Configuration.Abstractions from 1.1.0 to 1.0.2

Trying to run the application at this stage resulted in the following exception:

System.IO.FileLoadException: ‘Could not load file or assembly ‘Microsoft.Extensions.Configuration.Abstractions, Version 1.1.0.0 … The located assembly’s manifest definition does not match the assembly reference.

For some reason the migration had chosen version 1.1.0 for the Microsoft.VisualStudio.Web.BrowserLink dependency when it created the csproj file.

<PackageReference Include="Microsoft.VisualStudio.Web.BrowserLink" Version="1.1.0" />

The solution was to change it to 1.0.1 since allReady is targeting the LTS stream of ASP.NET Core currently. All of the other dependencies were registered with their LTS versions. This rouge 1.1.x dependency was therefore causing version conflicts.

<PackageReference Include="Microsoft.VisualStudio.Web.BrowserLink" Version="1.0.1" />

At this point, building the solution was getting a bit further. I now had a single warning left for the web project. This warning stated that AddUserSecrets(IConfigurationBuilder) is now an obsolete method.

‘ConfigurationExtensions.AddUserSecrets(IConfigurationBuilder)’ is obselete. ‘This method is obsolete and will be removed in a future version. The recommended alternative is .AddUserSecrets(string userSercretsId) or .AddUserSecrets<TStartup>()..’

The warning explains that we were now expected to pass in the user secrets key as a string to the method or use .AddUserSecrets<TStartup> which is syntactic sugar over AddUserSecrets(this IConfigurationBuilder configuration, Assembly assembly). Previously the storage location for the user secrets ID was the project.json file. It’s now moved to an assembly attribute. The previous overload is not guaranteed to return the correct Assembly so this could produce issues. You can read a fuller explanation from the team on the GitHub repo.

In our case I changed the appropriate line in Startup.cs from

builder.AddUserSecrets();

to

builder.AddUserSecrets("aspnet5-AllReady-468aac76-4430-43e6-848e-f4a3b90d61d0");

Test Project Changes

At this point the web project was building but there were issues inside the test project:

Test project warning after project.json migration

Found conflicts between different versions of the same dependent assembly that could not be resolved. These reference conflicts are listed in the build log when log verbosity is set to detailed.

Some of the dependencies we had been using had newer versions so I took a guess that perhaps I needed to update the Xunit / test related ones. I changed these three dependencies:

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0-preview-20170106-08" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.2.0-beta5-build1225" />
<PackageReference Include="xunit" Version="2.2.0-beta5-build3474" />

to

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
<PackageReference Include="xunit" Version="2.2.0" />

Finally the entire solution was building, tests were running and I could start the website locally.

Initial thoughts on csproj and Visual Studio 2017

Generally, everything seems to work once inside Visual Studio 2017. The migration process was a little more effort than I would have liked though.

No auto complete

In Visual Studio 2015 we had version auto complete for the dependencies from NuGet when editing project.json. We’ve lost this in Visual Studio 2017 which is a shame. It now pretty much forces me into the NuGet Package Manager which I find slower. I understand that autocomplete might come back at some point, but it’s a shame we’re lacking it upon release. 

Less easy to scan

Personally, I still prefer the project.json format for readability. There’s still a little more noise in the XML than we had inside the JSON version. That said, the work done to the revised csproj format is great and it’s a vast improvement on the csproj of the past.

Summing Up

Overall the process wasn’t too harrowing. Visual Studio and dotnet migrate handled most of the work for me. It was a shame that I had to resort on a few occasions to Google for manual solutions to solve problems that I’m sure should have been addressed by the tooling. The restore of the dependencies not working without clearing the cache was an unexpected problem.

But with the work of migration behind me I’m ready to move on. I’ve reached the acceptance stage on my project.json grieving process!

Anyone new to ASP.NET Core, using it for the first time in Visual Studio 2017 will probably look at older blog posts and wonder what we were all moaning about. The csproj is a great improvement on what we had in the past, so if you’ve never seen project.json, there’s nothing to miss and only improvements to enjoy.

ASP.NET Core Anatomy (Part 3) – UseMvc Dissecting and understanding the internals of ASP.NET Core

In parts one and two of this series I looked at the two main IServiceCollection extensions (AddMvcCore and AddMvc) in ASP.NET Core. These are used to add in the required MVC services when you plan to use the MVC middleware in your application.

The next thing you are required to do to enable MVC in your ASP.NET Core application is to include the UseMvc IApplicationBuilder extension from the Configure method of your Startup class. This method registers the MVC middleware into your application pipeline so that the MVC framework can handle requests and return the appropriate response (usually a view result or some JSON). In this post I will take a look at what happens when the UseMvc method is called during the application startup.

As with the earlier parts of this series, I’m currently using the rel/1.1.2 codebase from the MVC repository for my investigations. I’m still using the slightly older project.json based code as at the moment there don’t seem to be simple ways to debug into multiple ASP.NET Core source code assemblies in VS2017.

UseMvc is an extension on the IApplicationBuilder which takes an Action delegate of IRouteBuilder. The IRouteBuilder will be used to configure the routing for MVC. There is a more basic variant of the UseMvc method which does not required any parameters. This method simply calls down into the main version, passing in an empty delegate like so…

The main UseMvc method looks like this…

Let’s dissect what this method does. First it does a check to see if the IServiceProvider has an MvcMarkerService service registered in it. To do this it uses the ApplicationServices property which exposes the current IServiceProvider. Calling GetService on the service provider will try to locate a registered implementation for the requested Type. The MvcMarkerService is registered when AddMvcCore executes and as a result, if it is not found, it indicates that either AddMvc or AddMvcCore was not called during ConfigureServices. Marker services like this are used in quite a few places to help calling code verify proper registration of fundamental dependencies before they try to execute. The MvcMarkerService is just an empty class with no actual code.

Next, UseMvc requests a MiddlewareFilterBuilder from the ServiceProvider and sets its ApplicationBuilder using the IApplicationBuilder.New() method. The ApplicationBuilder creates and returns a new instance copy of itself when this method is called.

Next, UseMvc initialises a new RouteBuilder, setting the default handler to the registered MvcRouteHandler. At this point, the DI magic really kicks in and a bunch of dependencies start getting instantiated. MvcRouteHandler is requested and because it has some dependencies in its constructor, various other classes get initialised. Each constructor for these classes can also require additional dependencies which then also get created. This is a great example of how the DI system works. While all of the interfaces and concrete implementations are registered in the container during ConfigureServices, the actual objects are only created at the point they are required to be injected into something. By new-ing up the RouteBuilder, this snowball of object creation begins to occur, until all of the required dependencies have been constructed. Some of these are registered as singletons and will then live for the lifetime of the application, being returned whenever they are requested from the ServiceProvider. Others, might be constructed for each request or in some cases, constructed uniquely every time they are required. Understanding the details of how dependency injection works and specifically the ASP.NET Core ServiceProvider is out of scope for this post.

After creating the RouteBuilder, the Action<IRouteBuilder> delegate action is called which takes the RouteBuilder as its only parameter. In the MvcSandbox sample that I’m using to debug and dissect MVC we call the UseMvc method, passing in a delegate function via a lambda. This function maps a single route named “default” as follows.

This calls the MapRoute method which is an extension on the IRouteBuilder. The main MapRoute method looks like this:

This requests an instance of an IInlineConstraintResolver which is resolved by the ServiceProvider to a DefaultInlineConstraintResolver. This class takes an IOptions<RouteOptions> in its constructor.

The Microsoft.Extensions.Options assembly, as part of its work, calls into the MvcCoreRouteOptionsSetup.Configure method which takes a RouteOptions parameter. This adds a new constraint map of type KnownRouteValueConstraint to the Dictionary of constraint maps. This Dictionary is initialised with a number of default constraints when the RouteOptions is first constructed. I’ll look into the routing code in a little more detail in a future post.

A new Route object is constructed using the name and template provided. In our case this is added the RouteBuilder.Routes List. By this point, running the sample MvcSandbox application, our route builder now has a single route added.

Note that the MvcApplicationBuilderExtensions class also includes an extension method called UseMvcWithDefaultRoute. This method includes a call to UseMvc, passing in a hardcoded default route.

This route is defined with the same name and template as defined in the MvcSandbox application. Therefore it could have been used as a slight code saver within the MvcSandbox Startup class. For the most basic of MVC applications it’s possible that using this helper extension method might be enough for your application routing requirements. In most real-world cases I suspect you’ll end up passing in a more complete customised set of routes. It’s a nice shorthand though if you want to start with the basic routing template. It’s also possible to call UseMvc() without a parameter and instead use the attribute routing approach. 

Next, the static AttributeRouting.CreateAttributeMegaRoute method is called and the resulting route is added to the Routes List (at index zero) on the RouteBuilder. CreateAttributeMegaRoute is documented with the summary “Creates an attribute route using the provided services and provided target router.” and it looks like this:

This method, creates a new AttributeRoute which implements the IRouter interface. Its constructor looks like this:

In its constructor it takes an IActionDescriptorCollectionProvider, an IServiceProvider and a Func which takes an array of ActionDescriptor’s and which returns an IRouter. By default, with the AddMvc registered services, it will get an instance of ActionDescriptorCollectionProvider which is a singleton object registered with the ServiceProvider. ActionDescriptors represent the available MVC actions that have been created and discovered within the application. We’ll look at those another time.

The code (inside the CreateAttributeMegaRoute method) which creates the new AttributeRoute uses a lambda to define the code for the Func<ActionDescriptor[], IRouter>. In this case the delegate function requests an MvcAttributeRouteHandler from the ServiceProvider. Because this is registered as transient, the ServiceProvider will return a new instance of an MvcAttributeRouteHandler each time one is requested. The delegate code then sets the Actions property (an array of ActionDescriptions) on the MvcAttributeRouteHandler using the incoming ActionDescriptions array and finally the new handler is returned.

Back to UseMvc; it finishes by calling the UseRouter extension on the IApplicationBuilder. The object passed into UseRouter is created by calling Build on the RouteBuilder. This method will add the routes into the RouteCollection. Internally the RouteCollection keeps track of which routes are named and which are unnamed. In our MvcSandbox example we end up with a named “default” route and an unnamed AttributeRoute.

UseRouter has two signatures. In this case we’re passing in a IRouter so the following method is called:

This does a check for a RoutingMarkerService within the ServiceProvider. Assuming this is registered as expected  it adds the RouterMiddleware into the middeware pipeline. It is this middleware which will use the IRouter to try and match up requests to the controller and action within MVC which should handle them. The details of this process will feature in a future blog post.

At this point the application pipeline is configured and the application is ready to start receiving requests. That’s a good point to stop for this post.

Summary

In this post we’ve looked at UseMvc and seen that it sets up the MiddlewareFilterBuilder which will be used later. It then also gets an IRouter via a RouteBuilder and much of the work at this stage is around registering routes. Once everything is setup, the routing middleware is registered into the pipeline. It’s this code that knows how to inspect incoming requests and map the paths to the appropriate controllers and actions that can handle them.

Other Posts In This Series