.NET South East September 2017 Meetup With guest Jon Galloway

Last night was the second .NET South East meetup event and it was a special one. Through a bit of scheduling luck I was able to co-ordinate with Jon Galloway and make plans with him to join us for the evening; before returning with him to London today to attend the Progressive .NET conference.

Planning

After our first event we decided to slightly refine the entry and sign in process to make it easier for our Madgex volunteers to manage. We decided to sign in the attendees downstairs in the foyer with two volunteers – Chris and Martin – kindly helping to do that. We also moved our networking area (where we provided snacks, soft drinks and beer) into our office entrance area. There was a little more space for people to talk and it made it easier for Trev, another volunteer on the evening to watch for anyone trapped in the corridor when returning from the toilets!

As expected, we had major interest in this evening, with the RSVP limit of 60 reached in about 24 hours of announcing the event. With a few days to go our RSVP list had grown to around 14 people and it was clear we had a good chance of full attendance. From the last event we knew that 60 would be pushing the available space to its maximum. This presented a slight concern as we were conscious that whilst we wanted to accommodate everyone who arrived on the night, we might have people showing up who were not on the list. We had announced beforehand that to be as fair as possible we would have to limit entrance to those on the RSVP list. In the end we had a few last minute cancellations and no-shows so our numbers were about perfect.

The Event!

On the day of the event, Madgex had held a company meeting in the space we would be using for .NET South East. This was quite handy since it meant most of the chairs were already out and setup. After the company meeting I spent the next hour making sure everything was neat and in order. As with the last event, time seemed to disappear very quickly. This time the Madgex staff decided to order pizza which was rapidly consumed before the event began. 

At 6pm I headed out to meet Jon at his hotel, just around the corner from our office. It was great to finally meet Jon in person after chatting back and forth for a little while. We walked back to the office together and ran through the rough plan for the timings of the evening. Once in the office we did a quick technical check to make sure Jon’s laptop would output on the large TV we use for the event.

This month, I had decided to bring in my dSLR camera (5D Mark II) and a couple of lenses for the evening. Outside of developer work, my wife and I are (occasional) semi-pro photographers and so have a lot of equipment from photographing weddings etc. The photographic results from the first meetup were a little poor due to the low lighting. Despite our best efforts, most were either noisy, blurry or both. I hoped that by bringing in the pro gear I could hopefully get a few more shots to use to promote future events. I’ve yet to have time to check and process the photographs from the event. The ones included here are from my phone. I’ll add more at a later date.

In the end we had 53 attendees (including myself) and filled every seat in the room. A massive thanks again to everyone who attended the event. It was great to see some new faces among those returning after the first meetup in August. I’m terrible with names and faces so apologies to anyone I had forgotten from the first event!

Intro and News

Just after 7pm we gathered everyone in the conference room, ready for the event. I opened the evening with my introduction, including thanking our fantastic sponsors and then went on to discuss some of the news items I had gathered for this month.

SignalR Core Alpha

The first topic I touched on was the upcoming release of the first alpha of SignalR Core. I was a little premature in this news item as technically the announcement is not due until later this week. I was basing this news item from a pre-announcement-announcement by Damian Edwards during the ASP.NET Core community standup. This is SignalR re-imagined and re-written on top of ASP.NET Core 2.0. SignalR is a real-time framework to support cases where you need to send messages from the server to connected clients in real-time. Application can include classic chat scenarios, gaming, real-time dashboards and notifications.

To learn more about what the team have been working on you can view this video from Microsoft Build 2017.

Source Linking

The second news item I focused on was the ASP.NET Core support for Source Link, a feature which now enables us to debug into the ASP.NET Core source from our application. This is a useful feature, particularly for learning more about how ASP.NET Core works under the hood. I shared details of my early use if this feature, along with a few items I could not get working. I mostly referred to the information I’d collected for my Debugging ASP.NET Core 2.0 Source blog post.

.NET Standard 2.0 and .NET Framework 4.6.1 Issue

The final item I decided to include was some information about an issue with .NET Standard 2.0 with .NET Framework & NuGet as documented in this GitHub Issue. The short summary of the issue is that in some cases, when consuming a .NET Standard 2.0 library from a .NET 4.6.1 project there can be runtime crashes or compile time warnings due to the handling of dependent assemblies. This stems from two possible causes, the possibility of missing binding redirects and/or entire missing binaries. The team are aware of the causes and are working to address this in Visual Studio 15.4. In the issue Immo Landwerth provided some workarounds that might help in the interim.

.NET Conf

I also mentioned the .NET Conf event again, which is running in September. It’s a free, streamed conference organised by Microsoft which include a lot of great .NET Core 2.0 and ASP.NET Core 2.0 content from experts in their field.

I hope the news content is adding some value to the monthly events. I didn’t feel that my presentation was quite as polished as my first time. Partly due to having to try and consume and remember the key points of the items, with only a little time to absorb it. The tough thing with news is that it’s, well, new! So pulling together a few cohesive points and then talking about them without rehearsing is not that easy for me. Maybe I can convince a sponsor to provide a teleprompter for the next one!

Talks

With such a rare opportunity to have someone directly linked to the ASP.NET team join us we took full advantage, with Jon kindly presenting two talks during the evening.

Jon Galloway – What’s new in ASP.NET Core 2.0

Jon Galloway - Introducing ASP.NET Core 2.0

In this talk, Jon managed to pack in a summary of the many new ASP.NET Core 2.0 features. It was a demo packed hour as Jon discussed and showed examples of the improvements since the 1.x release.

Jon started with a quick overview of .NET Standard 2.0 and why this is an important feature for developers, providing better compatibility of libraries between the different platforms. Jon likened the .NET Standard to HTML specs which is a good analogy to understand why we need a standard. Jon gave an example of Keen.io who develop an event data platform. They were able to take their existing PCL libraries and convert them to .NET Standard 2.0, simplifying code sharing between their platforms.

Jon went on to talk about the new AspNetCore.All metapackage which takes advantage of the runtime store which now ships with the .NET SDK. This store provides pre-jitted, optimised versions of the packages which any application using the .All meta-package will take advantage of. Jon also demonstrated the simplification and changes in the Program.cs and Startup.cs files.

Next was a big new feature in ASP.NET Core 2.0, Razor Pages. This is a new pattern built on the existing MVC system which aims to simplify and remove some of the ceremony of the traditional MVC pattern for less backend-heavy pages. It’s a feature I really need to spend some time taking out for a spin.

Jon then showed some of the additional templates we have available for ASP.NET projects, such as Angular and React. These stem from Steve Sanderson’s work on Javascript Services.

Jon concluded with a big slide of smaller features and improvements that all made it into ASP.NET Core 2.0. It was an information packed session and really great to have all of the key changes made easily accessible to the audience. We had a good mix of developers, some who had used ASP.NET Core and some who had not yet had chance to work with it. This talk had something for everyone.

Jon Galloway – The .NET Foundation

Jon Galloway - .NET Foundation

In Jon’s second talk the theme was the .NET Foundation, of which Jon is the Executive Director. In this 45 minute session, Jon took us through some of the goals of the .NET Foundation and explained the work they do with open source projects.

First Jon talked through some of the phases of open source contribution for developers. This was a scale from the fear and uncertainty that developers and business can feel about even using and consuming open source, up to fully embracing it, sharing and maintaining open source projects with public roadmaps. More details about these stages can be found at A Wider NET.

Jon then detailed at a high level some of the phases of Microsoft’s own adoption of open source, from the earlier toe in the water with some source open code (read, don’t use or change) through to where they are today, developing out in the open with .NET Core, ASP.NET Core and even things like the documentation and the C# language specifications.

A small part of the .NET Foundation is to act as an ombudsman between open source projects and consumers to try and ensure that the project is acting in the interests of its users and responding appropriately to legitimate issues. A much larger part is the support the .NET Foundation can offer to projects in the form of sponsorship as part of the community project membership. This is not about monetary funding, but about providing advice and services to the projects. This can include things such as helping set up wildcard certificates or providing some Azure credits for example. The goals here are to help make sure the projects continue to thrive and do not die out. This is a very important thing for consumers of the libraries, particularly in enterprise situations, that may rely on these projects in their code. The foundation does not run the projects but works to support them as much as possible. Jon likened this to Open Source as a Service!

Jon also talked about the recent .NET Summer Hackfest, a 6 week event aimed at getting new contributors into open source with support from some great projects such as Brighter and Humanitarian Toolbox. Jon hopes to makes this an even bigger event next year. Contributing to Open Source for the first time can be a daunting prospect and the idea of bringing forward some friendly projects who can put together some smaller, bite-sized issues that new contributors can pick up is a great one.

It was really great to hear about what the .NET Foundation is doing to support the .NET ecosystem and to help the community open source projects to thrive and be sustainable for the long term.

A bit thanks to Jon for fighting through the jet lag to present these two great talks and for making the arrangements to come straight down to Brighton from the airport.

Prize Draws

With the end of the evening closing in, before heading off to the pub we drew the winners of the prizes from our fantastic sponsors for the event. The prizes we had to offer were:

JetBrains

One year individual subscription to any single JetBrains Toolbox product

Progress

  1. UI for Xamarin license code
  2. T-Shirts x 5

Manning

  1. ebook of choice

elmah.io

6 months Small Business license

Last time I draw names from a bowl. A bit low tech and also time consuming to print and cut up all of the names. Dan Clarke, fellow .NET user group leader has created a desktop app which randomises winners from the Meetup.com RSVP list. He has kindly provided the application code on GitHub. For this event I decided to give that a go. It worked quite well and simplified things nicely, although I was still conscious that at the end of the evening, it takes a little time to draw all of the prizes. One big advantage is that it stores the winning member ID so we don’t have to worry about duplicate names causing us to require some kind of tie-break. I am pondering whether for the next event I pre-draw the names using the app to save some time.

The rules as with the last event were:

a) names are added from the RSVP list (as at about 1 to 1.5 hours before the event)
b) if the name drawn is not in attendance, we redraw.

Next events

We have some great speakers lined up for the next couple of months, and I’m working with a few people of plans for the next couple of months after that.

.NET South East October 2017 – Rabeb Othmani
Rabeb joins us with her talk “Welcome to the age of conversational interfaces” – looking at how we can build interfaces using SMS, voice and bots.

.NET South East November 2017 – Michael Newton
Michael joins us with his talk “Making Distributed Systems in .NET Easier” – discussing distributed architecture with .NET.

Call for speakers

I’d love to get a range of varied content and speakers to present at our user group. We have a nice pipeline for the coming months but those months will fly by very quickly. If you’d be interested in speaking at a future event we’d love to have you. Please get in touch via the contact form on this blog or ping me on Twitter and we can discuss availability and topics.

I’m really keen to draw as many speakers from our local community too so please let me know if you might be interested in speaking. Perhaps you have presented a talk internally and could open it up to a wider audience. I highly recommend speaking as a way to develop professionally. I’m happy to offer advice for new speakers and help where I can.

Links

Some links from this months event:

Debugging ASP.NET Core 2.0 Source Code Using ASP.NET Core symbols with Source Link support

In the early days of ASP.NET Core 1.0, before Visual Studio 2017, back when we had the project.json project format, we were able to take advantage of a nice feature to enable source debugging of referenced libraries and code. We could add the paths of cloned source repositories into our global.json file and these would then be preferred over Nuget when available.

I used this quite extensively when writing some of my posts for my MVC Core anatomy series (something I hope to get continue with at some stage). I blogged about how we could set this up in this previous post – Debugging into ASP.NET Core Source. Unfortunately with the switch back to MSBuild and csproj, we lost the ability to easily debug through the ASP.NET Core source files.

Yesterday during the ASP.NET Community Standup Jon Galloway highlighted a tweet by David Fowler regarding new ASP.NET Core source linking support. Damian Edwards went on to provide some more detail about this new feature and afterwards I decided to take a quick look at it myself. I expect this post to serve as an early introduction to source linking and I will hopefully blog on more detailed elements once I’ve had time to explore them more fully. For this post we’ll focus on how we can get started with debugging into the ASP.NET Core source using source linking.

What is Source Linking?

Let me caveat this explanation with the fact that symbol files are not something I’ve previously messed around with besides knowing that PDB files existed. I’m coming at this blog relatively fresh. I’ll be explaining things as I’ve understood them so far and will happily make corrections and updates if necessary.

Like me, you may have noticed PDB files being created under some circumstances when compiling your code. These files hold the symbol information which can optionally be used to support debugging into external source. Some types of symbol files may contain some of that source code or mappings to the source code.

For a long time Microsoft have hosted Symbol servers which hold published symbol files for the Microsoft products such as .NET Framework and ASP.NET Core. Visual Studio supports downloading symbols dynamically. To do this you must disable the “Enable Just My Code” option in the Debugging > General options. By default this option is enabled in Visual Studio.

Enable Just My Code Option

For more information on Symbols, Symbol Servers etc see this MSDN link.

Source linking allows you to embed a manifest inside the symbol file. For a given method name and location in the method that is being called it can identify what file contained the code and where it can be retrieved from. The ASP.NET Core libraries (not .NET Core currently) now support Source Linking and provide links to the code which is hosted on GitHub.

Enabling and Using Source Linking

The first requirement is that you are running Visual Studio 2017 on the latest update (15.3) which added Source Link support. With this installed if you check the Debugging > General options you will see Source Link enabled.

Enable Source Link support

As well as ensuring “Enable Just my Code” is not checked you must also enable the Microsoft symbol servers. In the Debugging > Symbols options you can check the “Microsoft Symbol Servers” from the list of symbol file locations.

Enable Microsoft Symbol Servers

When enabling the symbol servers you will need to accept the possibly the performance impact that it may introduce when debugging.

Symbol Server Performance

We are now setup and ready to debug into the ASP.NET Core source. To test this I created a default ASP.NET Core 2.0 MVC project inside Visual Studio. I then added a break point to the Index action on the HomeController. I then started debugging the application. The first time when debugging you may see messages like this in the status bar.

Loading Symbols

This is the symbol files being downloaded and may take a short while to complete.

Once you application is running and the breakpoint in your code is hit you can navigate down the call stack to see all of the external ASP.NET code that is being executed.

Call Stack Source Link

 If you double click any of these calls the editor will use the symbols to determine where that code is located for the frame. Using the link inside the symbols file, Visual Studio will download the source file from GitHub. When Source Link needs to download source you will see a warning dialog like this:

Source Link Dialog

You can chose the first option to download for this specific source file and continue to debug using that file. If you chose the first option you will see this dialog for each new source file that is required. You can select the second option instead which will download the file and disable the warning for future files.

Now that we have the source it will be displayed at the appropriate location from the frame you selected.

Source Link External Code

Now that we have the source file, you can also add your own breakpoint somewhere else within that file which will then be set to be hit when debugging your application. Even if we stop debugging and start again, this still seems to be hit successfully.

External Source Breakpoint

I was also curious about how we could set breakpoints in other parts of the code, without having to rely on access the source via the call stack. Damian mentioned a feature I’d never used where we can set a breakpoint manually from the breakpoint window. I have attempted to set a new function breakpoint up to one of the ASP.NET Core methods but so far I haven’t been able to get it working as Visual Studio states that the source is not available. I will pursue this and hopefully include details in a future post.

Summary

It’s great to see the beginning of easier debugging of external source coming to ASP.NET Core. Already there is value that I can gain from this feature to allow me to debug into the ASP.NET Core source to understand the internal workings. It’s not a sweet as what we were able to do back in project.json days but is certainly a step forward since we went back to csproj and MSBuild.

I miss the simplicity of cloning the full source for a repository and being able to navigate through it and add breakpoints to the external code. The source linking mechanism is good for more specific debugging cases where you want to dive into the external call stack. At the moment, once you have the source cs file via Source Link, you can’t navigate to other methods so exploring the code and setting further breakpoints is not possible outside of that file. If I can get the manual breakpoints working then that will be slightly better as at least I can view the source and determine methods I might want to break on, then set those up manually.

Chatting to David Fowler on Twitter I also understand that more features are being planned so I’ll be watching for those with interest. In addition, work is underway to get this supported by other non-Microsoft OSS projects such as Xunit and maybe in the future, JSON.NET. This repository (which I’ve not dug through yet) provides some build tools which can help creating source link symbols. I will also be looking at this more in the future.

Further References

Portable PDB files

Source Link

Upgrading to ASP.NET Core 2.0 My experience of upgrading a real-world solution from ASP.NET Core 1.0 to 2.0

On the 14th of August, Microsoft announced the release of .NET Core 2.0, ASP.NET Core 2.0 and EF Core 2.0, the next major releases of their open source, cross platform frameworks and libraries. This is a very exciting release and one which I hope marks the stabilisation of the framework and enables more developers and businesses to begin really looking at using .NET Core 2.0.

One of the big changes with .NET Core 2.0 is support for the new .NET Standard 2.0 specification (also part of the release announcements) which defines the API surface that platforms should conform to. This brings back around 20,000 APIs that were not originally included in .NET Core 1.x. This should mean that porting existing full .NET Framework applications over to Core may now be a more realistic prospect with much greater parity between the frameworks.

As I have discussed a few times on this blog, I contribute to a fantastic project called allReady, managed by the charity, Humanitarian Toolbox. This project started originally in the early beta days of .NET Core and ASP.NET Core and has evolved along with the framework through the various changes and refinements. With the release of 2.0 we were keen to upgrade the application to use .NET Core 2.0 and ASP.NET Core 2.0. I took it upon myself to attempt to upgrade allReady and to document the experience as I went. Hopefully I’ve found the right balance of detail to readability for this one which has been a bit of an epic!

Installing .NET Core 2.0

The first step you will need to complete is to install the new 2.0 SDK and if you use Visual Studio as your IDE of choice, you will also need to install the latest version of Visual Studio 15.3.x in order to work with .NET Core 2.0. These steps are well documented and pretty easy.

Upgrading the MVC Project

Upon loading the allReady web solution in Visual Studio 15.3 (aka 2017 update 3), my first focus was on upgrading the web project and getting it to run. I therefore unloaded the test project so that I wasn’t distracted by errors from that.

Many of the main steps that I followed as I upgraded the solution can be found outlined in the Microsoft .NET Core 1.0 to 2.0 migration guide.

Upgrading the Project File and Dependencies

The first job was to upgrade the project to target .NET Core 2.0 and to upgrade its dependencies to request the ASP.NET Core 2.0 packages. To do this I right clicked my project and chose to edit the csproj file directly. With .NET Core projects we can now do this without having to unload the project first. .NET Core projects have a targetFramework node which in our case was set to netcoreapp1.0. To upgrade to target the latest Target Framework Moniker (TFM) for Core 2.0 I simply changed this to netcoreapp2.0.

Our project file also included a runtimeFrameworkVersion property set to 1.0.4 which I removed to ensure that the project would use the latest available runtime. The migration guide also specifies that the PackageTargetFallback node and variable should be renamed to AssetTargetFallback and so I made that change.

The next big change was to begin using a new ASP.NET Core meta package to define our dependencies. One of the drawbacks that people have experiences with depending on the many individual Nuget packages which make up ASP.NET Core platform is that management of the package versions can be a bit painful. Each package can have slightly different minor version numbers as they revision separately. During a patch release of ASP.NET Core for example, it can be hard to know which exact versions represent the latest of each of the packages as they don’t necessarily all update together.

The ASP.NET team are hoping to solve this with the availability of a new Microsoft.AspNetCore.All metapackage. This package contains dependencies to all of the common Microsoft.AspNetCore, Microsoft.EntityFrameworkCore and Microsoft.Extensions packages. You can now reference just this package to enable you to work with all of the ASP.NET Core and EF Core components.

One of the changes that enables this is in the inclusion of a .NET Core runtime store which contains all of the required runtime packages. Since the packages are part of the runtime, your app won’t need to download many tens of dependencies from Nuget. The runtime store assets are also precompiled which helps with performance.

To make use of the new meta package I first removed all existing ASP.NET related dependencies from my explicit project package references. I could then add in the following reference: <PackageReference Include=”Microsoft.AspNetCore.All” Version=”2.0.0″ />. 

The final change in the project file was to update the versions for the .NET Core CLI tools specified in the DotNetCliToolReferenence nodes for our project file. In each case I moved them to the 2.0.0 version. With this completed I was able to save and close the project file, which triggers a package restore.

Our project file went from this:

to this:

The next thing I needed to do was to remove a global.json file that we had in our solution which was forcing the use of a specific SDK version; in our case 1.0.1. We want our project to use the latest SDK so I removed this file entirely. At this point I was in a position to attempt to compile the web project. As expected the build failed and a number of errors were listed that needed to work through fixing.

Identity / Authentication Changes

With ASP.NET Core 2.0, some of the biggest breaking changes occur in the Identity namespace. Microsoft have adjusted quite a few things regarding the Identity models and authentication. These changes did require some fixes and restructuring of our code to comply with the new model. Microsoft put together a specific migration document which is worth reviewing if you need to migrate Identity code.

The first change was to temporarily comment out some code we have as an extension to the IApplicationBuilder. I would use this code to ensure I had fully replicated the required setup before removing it. We used this code to conditionally “use” the various 3rd party login providers within our project; for example – UseFacebookAuthentication. One of the changes made with Identity in ASP.NET Core 2.0 is that third party login providers are now configured when registering the Authentication services and are no longer added as individual middleware components.

To account for this change I updated our ConfigureServices method to use the new AddAuthentication extension method on the IServiceCollection. This also includes extension methods on the returned AuthenticationBuilder which we can use to add and configure the additional authentication providers. We conditionally register our providers only if the application configuration includes the required App / Client Id for each provider. We do this with multiple, optional calls to the AddAuthentication method. I’ve checked and this is a safe approach to meet this requirement. At this point I could replicate the 3rd party authentication configuration that we had previously setup using the UseXYZAuthentication IApplicationBuilder extensions.

With this complete, our Configure method could be updated to include the call to UseAuthentication which adds the authentication middleware. The commented code could now be removed.

IdentityCookieOptions

Our account controller (based on the original ASP.NET Core MVC template) had a dependency on IOptions<IdentityCookieOptions> to get the ExternalCookieAuthenticationScheme name. This is now redundant in 2.0 as these are now available via constants and we can use that constant directly in our login action as per the authentication migration guide.

In 1.0 we set our AccessDeniedPath for the cookie options as one of the options on the AddIdentity extension for the IServiceCollection. Where we previpusly set it as follows:

There is now a specific extension to configure the application cookie where we set this value so I added that code to ConfigureServices.

The next change is that IdentityUser and IdentityRole have been moved from the Microsoft.AspNetCore.Identity.EntityFrameworkCore namespace to Microsoft.AspNetCore.Identity; so our using statements needed to be updated to reflect this change in any classes referencing either of these.

Next on my build error hit list was an error caused by Microsoft.AspNetCore.Authentication.FailureContext no longer being found. This has been renamed to RemoteFailureContext in ASP.NET Core 2.0 so I updated the affected code.

Another change as part of Identity 2.0 is that the Claims, Roles and Login navigation properties which we made use of have been removed from the base IdentityUser class. As a result I needed to add these back into our derived ApplicationUser class directly and update the OnModelCreating method inside our DbContext to define the correct foreign key relationships. This was as described in the migration guide for Authentication and Identity.

A small change I had to take care of is that GetExternalAuthenticationSchemes has been made Async (and renamed accordingly) so I updated our code to call and await the GetExternalAuthenticationSchemesAsync method – The return type has also changed, so I also needed to update one of our view models to take the resulting list of AuthenticationSchemes rather than AuthenticationDescriptions.

The final authentication change was the result of a new set of extension methods being added to HttpContext in Microsoft.AspNetCore.Authentication. These are intended to be used for calling the SingOutAsync and similar methods which were previously available via the IAuthenticationManager.

In places where we called these I changed from

await httpContext.Authentication.ChallengeAsync();

to

await httpContext.ChallengeAsync();

Other Changes / Build Errors

With the authentication and Identity related changes completed I still had a few build errors to take care of before the application would compile.

In 1.1.0 Microsoft added an additional result type of AcceptedResult (the issue is available here) and a helper method on ControllerBase to easily return this result. Since we had been target 1.0.x we had not faced this change before. Our SmsResponseController was exposing a constant string called “Accepted” which then hid the new inherited member on ControllerBase. I renamed our member to avoid this naming conflict.

We also found that Microsoft.Net.Http.Headers.ContentDispositionHeaderValue.FileName had changed from being defined as a string to a StringSegment instead. This meant we had to update code which was calling Trim on it to first call ToString on the StringSegment value.

In one place we were using a previously available TaskCache.CompletedTask to get a cached instance of a completed Task. However, since Task.CompletedTask is now available due to targeting NetStandard 2.0 this had been removed so our code could switch to using Task.CompletedTask instead.

Other Migration Changes

There are some other structural changes we can and should make to an existing ASP.NET Core 1.x project to take advantage of the ASP.NET Core 2.0 conventions. The first of these was to update program.cs to use the newer CreateDefaultBuilder functionality. This method is designed to simplify the setup of an ASP.NET Core WebHost by defining some common defaults which we previously had to setup manually in the Startup class. It adds in Kestrel and IISIntegration for example. The IWebHost in 2.0 now also sets up configuration and logging, registering them with DI earlier in the application lifecycle. The defaults work for basic applications but depending on your requirements you may need to use the ConfigureLogging and ConfigureAppConfiguration methods to apply additional setup of these components.

Out program.cs changed from:

to

Now that Configuration and Logging are setup on the IWebHost, we no longer need to define the setup for those components in the Startup.cs file, so I was able to strip out some code from Startup.cs. In 1.x we used the constructor of Startup to use the ConfigurationBuilder to setup Configuration. This could be taken out entirely. Instead we could ask for an IConfiguration object in the parameters which will be satisfied by DI as it is now registered by default.

I was also able to remove the logging setup which used an ILoggerFactory in the Configure method in 1.x. This is now also setup earlier by the IWebHost which feels like a better place for it. It also means we get more logging during the application bootstrapping. One change I made as a result of relying on the defaults for the logging setup was to rename our config.json file to appSettings.json. appsettings.json is included by default using the new CreateDefaultBuilder so it’s better that our config file matches this convention.

Finally, ApplictionInsights is now injected into our application by Visual Studio and Azure using a hook that lets them place code into the header and body tags, so we no longer need to manually wire up the ApplicationInsights functionality. This meant I could strip the registration of the service and also remove some code in our razor layout which was adding the javascript for ApplicationInsights.

From out ConfigureServices method I removed:

services.AddApplicationInsightsTelemetry(Configuration);

From our _ViewImports.cshtml file I removed

@inject Microsoft.ApplicationInsights.AspNetCore.JavaScriptSnippet JavaScriptSnippet

From the head section of our_Layout.cshtml file I removed

@Html.Raw(JavaScriptSnippet.FullScript)

Partial Success!

At this point the code was able to compile but I hit some runtime errors when calling context.Database.Migrate in our Configure method:

“Both relationships between ‘CampaignContact.Contact’ and ‘Contact’ and between ‘CampaignContact’ and ‘Contact.CampaignContacts’ could use {‘ContactId’} as the foreign key. To resolve this configure the foreign key properties explicitly on at least one of the relationships.”

And

“Both relationships between ‘OrganizationContact.Contact’ and ‘Contact’ and between ‘OrganizationContact’ and ‘Contact.OrganizationContacts’ could use {‘ContactId’} as the foreign key. To resolve this configure the foreign key properties explicitly on at least one of the relationships.”

To solved these issues I updated our DbContext fluent configuration in OnModelCreating to explicitly define the relationships and foreign key.

From:

To:

This got me a step further but I then hit the following error:

System.Data.SqlClient.SqlException: ‘The name “Unknown” is not permitted in this context. Valid expressions are constants, constant expressions, and (in some contexts) variables. Column names are not permitted.’

I tracked this down to a migration which sets a default value on an integer column using an enum. I found that I needed to explicitly cast the enum to int to make this migration work as expected.

Another step forward; but I still ran into issues. The next error I received was System.ObjectDisposedException: ‘Cannot access a disposed object.’ from Startup.cs when calling await SampleData.CreateAdminUser();

This was caused by a naughty use of async void for the Configure method. I removed the async keyword and used GetAwaiter().GetResult() instead since async void is not a good idea!

By this point I was really hoping I was getting somewhere. However next I had some odd issues with our TagHelpers. We have two tag helpers used to aid some datetime functionality. The errors I was seeing seemed to be due to the TagHelpers getting invoked for the head and body elements of the page. I’ve yet to spend enough time to track down what causes this so have applied workarounds for now.

On our TimeZoneNameTagHelper we were getting a null object error when this tried to apply for the head tag. We expect a TimeZoneId to be supplied via an attribute which was not present on the head tag and so this resulted in null TimeZoneId when we tried to use it to lookup the time zone with FindSystemTimeZoneById. The temporary fix in this case was to check the TimeZoneId for null and just returning if so.

With our TimeTagHelper I had to do an explicit check within the Process method to ensure the TagName matched “time”. This avoided it being applied for the head and body tags. I have created follow-up issues to try to understand this behaviour.

With these changes in place, the code was finally compiling and running. Yay!

Upgrading the Test Project

With the main web application working I was ready to focus on upgrading the test project and making it compile (and seeing if the tests would pass!) The first step here was updating the project file to target netcoreapp2.0 as I had done with the web project. I also updated some of the dependencies to the latest stable versions. This was partially required in order to restore packages and it also made sense to do it at this point since I already had a lot of changes to include. Some of our dependencies were still old pre RTM packages. I also took the chance to clean out some unnecessary nodes in the project file.

With the packages restoring, attempting a build at this stage left me with 134 build errors! Some as a result of changes to Identity, some due to upgrading the dependencies and some due to code fixes made to the main project as a result of the migration.

The first broken tests I focused on where any that had broken due to the Identity changes. These were relatively quick to update such as fixing changes namespaces.

to

I then had a number of tests which were broken due to a change in Moq, the library we use for mocking objects in our tests. When setting up methods of mocked objects we could previously return a null quite simply passing null as the parameter to ReturnsAsync. However there is now another extension method also accepting a single parameter and the compiler is not sure which one we are intending to use. So this now requires that we explicitly cast this as a null of the correct type to indicate we are passing the expected value and not a delegate which returns the value. This resulted in me having to update 46 tests.

The remainder of build failures were mostly caused by changing the number of parameters for the AccountController constructor so our tests which were creating one as the subject under test needed to be updated also to match the correct number of parameters.

At this point I had compiling test code and I was then able to run my tests! Oh, 101 failed tests!

When I looked a little deeper I noticed these were nearly all tests which used our InMemoryContextTest abstract base class which includes a registered instance of an InMemory DbContext on an IServiceProvider. With a bit of trial and error I realised that my queries were not returning any results, where previously they had in 1.o. When I experimented I found that it was in cases where our query called Include to eager load some of the related entities. However, our seed data for the test which populated the InMemory database for each test had not set those related entities. The InMemory provider does not enforce referential integrity and so there are no errors thrown when saving objects with missing required navigational properties.

In 1.x the query behaviour worked under this scenario but in 2.0 something had changed. I raised an issue about this one and the EF team responded quickly with… “The reason for the behaviour change is that now include is using navigation rewrite logic to construct the queries (whereas before we manually crafted include statements). Navigation rewrite produces INNER JOIN pattern for required relationships and LEFT JOIN pattern for optional. Before we would always hand-craft LEFT JOIN pattern, regardless of the relationship requiredness between child and parent.”

To correct for this I needed to ensure our test setups added the required related entities so that they would be returned from the queries as expected. In our actual code, running using the SqlProvider this is not an issue since the saves enforce the referential integrity.

With the tests fixed up I was finally at a point where everything compiled, ran and the tests were passing. I considered this a good place and was able to submit my PR to get the allReady project to 2.0 which was promptly merged in.

Summary

For the most part the migration documentation provided by Microsoft where very good and covered many of the things I actually experienced. In a few cases I found little extra things I needed to solve. For the most part the issues were around the tests and the EF changes probably took longest to isolate and then fix-up. It’s great to have been able to help move the project forward and get it to 2.0 very soon after release. It’s a great reference project for developers wanting to view (and hopefully work on) a real-world ASP.NET Core 2.0 solution. Hopefully my experience will help others during their migrations.

.NET South East August 2017 Meetup With speakers Dylan Beattie and Steve Gordon

Last night we held the first ever .NET South East meetup event! I’ve been really looking forward to this meetup since first announcing the idea at the end of June. I talked about some of my motivations behind starting the group in an earlier blog post.

I’d spent a lot of time leading up to this event trying to think about all of the bits I needed to plan and have ready. I had great support from some of the other community leaders to help with ideas, advice and suggestions. I was also able to attend an event at Microsoft in London for community leaders, speaking with some of their team about how they can support user groups, so I’m looking forward to working with them too.

Over the days and weeks before the event I had been staggered by the number of RSVPs we were getting via meetup.com. One of my two big concerns when moving forward to launch the group had been whether there would be interest from the local community and people would show up. I had set a rather arbitrary 60 person limit for the group, never really expecting to hit that. However, with a day or two left before the event, we were full! In fact I was starting to worry that if everyone turned up, we would run out of space and seating.

The big day!

Thoughts of the meetup were always in the back of my mind during the day and as the start time approached I was equal parts excited and nervous. I start and finish early at work so by 4pm I was able to shoot out and grab an early dinner from Pompoko in Brighton. It was nice to have 20-30 minutes to relax out of the office and prepare myself for the evening ahead.

Returning to the office at 4:30pm it was time to begin setting up. We have 3 meeting rooms at Madgex which can all be opened up into one large space. It’s the perfect location for an event like this as we have a large TV screen for presenting on and an audio system with an array of microphones. I’m hugely thankful to two Madgex staff in particular at this stage who helped me to get this room ready and setup the equipment. Leah our amazing office administrator was on hand helping to set up the seating, whilst Ricky our IT tech was there to ensure the audio/visual side was all functioning as expected.

Our one and only technical hitch for the evening was actually with one of the folding doors which allow us to open up the last room as part of the space. The last folding section was jammed shut and we couldn’t open the door fully. However, it wasn’t a major issue and we were still able to get the seating setup. It was not one of the things I’d worried about going wrong! I’d been mostly concerned with the TV output and microphones working correctly.

Madgex venue for .NET South East

I was amazed at how quickly the time evaporated as we got the room and snacks prepared. Once the meeting rooms were ready I set about putting up some signs to guide people into the Madgex offices. By 6:15pm we had our first early bird arrivals. Some more of my colleagues at Madgex jumped in to add their support here, helping get people through the security doors and into the office. Madgex are on the 1st floor of a shared building and access is tightly controlled. To access the building you need to be buzzed in, then to use the lifts you will need an access fob.

It was actually the logistics of this which was the hardest part of the evening. Fortunately I had two volunteers on hand to help. Rachel, our development team lead kindly based herself in the foyer of the building to let people in. She would then put them in the lift and swipe her access fob so they could be delivered to the 1st floor. Ready and waiting in the entrance of the Madgex office, Chris, one of our senior developers was ready to greet the guests and get them signed in. With RSVP lists on hand Chris was able to tick off the attendees for the evening.

The other logistics challenge we have is access to the toilets, or more specifically, how people get back into the Madgex office. Once you leave for the toilets a security locked door stands in your way if you want to return. We had organised visitor access fobs for the evening and Chris was superb as passing those out and gathering them back from our guests. During our 10 minute break Chris manned the door to enable people to use the facilities.

Without Chris and Rachel helping on the night I’m really not sure how we could have gotten everyone in so successfully, so I know for next time that I need to line up at least two volunteers again. We also realised that once we start it’s near impossible to hear the intercom buzzer so I have made sure to update our details on meetup to stress that entry after 7pm can’t be guaranteed. Unless we are able to get someone stationed near the door (who was not worried about missing the talks) I’m not sure how we can improve this. We’ll try to think about possible solutions to that problem, but hopefully everyone arrives on time. By having the arrivals from 6:30pm and talks at 7pm, we hopefully give enough of a window to get people into the event.

In the end we had 49 attendees (including myself) and I think nearly filled every seat in the room. I was really amazed by the turnout as people started to fill up the area where we were serving drinks and snacks. Before long it was getting quite congested. We’ll think about the possibility of picking a different networking space for the next event. I’d honestly not dreamed that the first event would be so popular. A big thanks to everyone who made time to attend and show their support. It’s great to see that we have such a large community who are willing and able to attend. I really hope we can keep the attendance level up for the upcoming events.

Audience at .NET South East

Intro and news

As we hit 7pm it was time to get everyone seated and begin the event. I expected to be more nervous than I was as I prepared to give my introduction. However, I felt pretty good and after a minute or so I was into my stride. After covering the obligatory health and safety notices I went on to share some of the reasons behind starting the group and welcoming everyone to the event. I then took a chance to thank our sponsors, especially Madgex for the support they’ve given and for providing a venue for the evening. The meeting rooms are a great space and I hope everyone was reasonably comfortable in there.

I also covered a little news and events section which originally I was unsure about including. However, with the release of .NET Core 2.0 last week I felt that was worth spending a few minutes to talk about it. The big changes are the wider API surface now available in .NET Core 2.0 which align it to .NET Standard 2.0. This hopefully eases the barrier to entry for companies with existing code that they may want to migrate over to core.

I also highlighted the .NET Conf event which is running in September as well. It’s a free, streamed conference organised by Microsoft which will likely include a lot of .NET Core 2.0 and ASP.NET Core 2.0 content.

Talks

Dylan Beattie: Life, Liberty and the Pursuit of APIness : The Secret to Happy Code

With my introduction complete it was time for our first talk. Dylan Beattie was kind enough to join us from London (on his birthday) to give his fantastic talk entitled: Life, Liberty and the Pursuit of APIness : The Secret to Happy Code. In this very entertaining talk Dylan presented examples of both good and bad interfaces and how these can affect the happiness and frustration of end users.

During the talk he highlighted the power of giving helpful error messages and prompts for the user/developer to solve the issue wherever possible. Blank wall error messages should be avoided where they offer no useful information to enable the user to proceed. The way those messages are phrased is also important.

Dylan spoke about a personal obsession of mine – including proper XML comments to supply tools like Visual Studio’s intellisence a way to provide developers with useful instruction about how to use your library / code. This is something I’m very keen on as I’ve used a few poorly commented libraries that provide no intellisense support to guide you through their API.

The discussion continued onto proper and relevant logging / monitoring and how logging levels and messages should be used wisely to provide insight into the health of systems. A good recommendation is leaving relevant debug logging in place that can be enabled in production if you need to diagnose hard to replicate errors.

It was a great talk and really well received by the audience.

Dylan Beattie speaking at .NET South East

Dylan ended with a short promotion for a .NET conference that he helps to organise in London called Progressive .NET. There’s a fantastic speaker line-up for the event so I recommend you check it out and convince your boss to send you along! We also have a 20% discount code you can use against the current list price: SE_PROGNET_20

My Talk: Docker for .NET Developers

After our 10 minute break I was pleased to see that nearly everyone had stayed for the second talk of the evening. This time I was in the spotlight and presenting my talk about how .NET developers can get started with Docker. In this talk I share our experience at Madgex as we got started with Docker for a new product. Along the way I explain the architecture we developed and how we used Docker to ease the workflow for our front end developers. Along the way I show some code demos about how we can get started using Docker, building images and running containers. We look at using docker-compose for co-ordinating multiple containers.

I conclude the talk with an explanation of how we’d developed a build and deployment process and how we run in production on AWS using Docker. This included a final demo showing the deployment process in action.

Steve Gordon speaking at .NET South East

Prize Draws

With the end of the evening closing in, before heading off to the pub we drew the winners of the prizes from our fantastic sponsors for the event. The prizes we had to offer were:

JetBrains

One year individual subscription to any single JetBrains Toolbox product

Progress

  1. DevCraft Complete license code
  2. T-Shirts x 5

Manning

  1. ebook – 1Docker in Action by Jeff Nickoloff
  2. ebook – Docker in Practice, 2nd edition by Ian Miell and Aidan Hobson Sayers

elmah.io

6 months Business license

I went for the low tech, names out of a bowl approach for the first event! The rules I’ve devised which I hope are fair are:

a) names are added from the RSVP list (as at 1 to 1.5 hours before the event)
b) if the name drawn is not in attendance, we redraw.

Congratulations to the winners. I hope everyone who won was happy with their prize. One issue we did encounter were that meetup.com doesn’t enforce full names for RSVP’s so we could run into issues with drawing a winner signed up with just their first name when more than one person in the room shares the name! I’m not sure what we can do there but we’ll try to manage it fairly or do some kind of tie break in those cases. I’ll also urge our attendees to add their full names when registering on Meetup.

Next events

One of the main concerns I had when starting the user group was finding speakers. So far those concerns have not been warranted and I’m pleased to have been able to line up some great speakers for the coming three months of events.

.NET South East September 2017 – Jon Galloway
We’re excited to announce that Jon Galloway from Microsoft will join us for the evening to share two exciting talks. What’s new in ASP.NET Core 2.0 and a talk about The .NET Foundation. This is filling up fast and we expect it to be quite popular. Please make sure you visit the link and RSVP to attend as we will have to limit numbers.

.NET South East October 2017 – Rabeb Othmani
Rabeb joins us with her talk “Welcome to the age of conversational interfaces” – looking at how we can build interfaces using SMS, voice and bots.

.NET South East November 2017 – Michael Newton
Michael joins us with his talk “Making Distributed Systems in .NET Easier” – discussing distributed architecture with .NET.

Call for speakers

I’d love to get a range of varied content and speakers to present at our user group. We have a nice pipeline for the coming months but those months will fly by very quickly. If you’d be interested in speaking at a future event we’d love to have you. Please get in touch via the contact form on this blog or ping me on Twitter and we can discuss availability and topics.

I’m really keen to draw as many speakers from our local community too so please let me know if you might be interested in speaking. Perhaps you have presented a talk internally and could open it up to a wider audience. I highly recommend speaking as a way to develop professionally. I’m happy to offer advice for new speakers and help where I can.

Links

A collection of links shared during the evening.

Dylan Beattie’s blog
Steve Gordon’s blog
.NET Conf – Sept. 19th – 21st
Progressive .NET Tutorials – Sept. 13th to 15th
Humanitarian Toolbox Summer Hackfest
.NET Core 2.0 / ASP.NET Core 2.0 Introduction – Scott Hunter
Ian Cooper – Creating a .NET Renaissance (NDC Oslo 2017)

Docker for .NET Developers slides
Docker for .NET Developers – Demo 1
Docker for .NET Developers – Demo 2
Docker for .NET Developers – Demo 3
August 2017 Meetup intro / summary slides

Implementing IHostedService in ASP.NET Core 2.0 Use IHostedService to run background tasks in ASP.NET Core apps

Update 30-08-2017: ASP.NET Core 2.0.0 is now released. I have updated my sample repo to 2.0.0.

I’ve had chance to play around with ASP.NET Core 2.0 preview 2 a little in the last few weeks. One of the things I was keen to try out and to understand a little better was the new IHostedService interface provided by Microsoft.Extensions.Hosting. David Fowler and Damian Edwards demonstrated an early example of how to implement this interface using the preview 1 of ASP.NET Core 2.0 at NDC Oslo. At the time of that demo the methods were synchronous but since then they have been made asynchronous.

Full disclosure: After taking a first pass at creating something using this interface I ran the code past David Fowler and he kindly reviewed it. As I suspected, I was not using it correctly! Since then David was kind enough to answer a few questions and even provided a sample of a base class that simplifies creation of hosted services. After my failure to understand the expected implementation of the interface and a general realisation that I needed to learn more about using Task cancellations with async/await, I almost decided to ditch my plan to write this blog post. However, I realised that this is still probably a good sample to share since others may run into the same mistakes I did. After speaking with David I believe this is appropriate use of the interface.

One of the challenges when starting out was trying to use something in preview that had no samples or documentation yet. While I hope no one will use this prior to RTM of 2.0 when I expect full documentation will be made available, I’m sure people may benefit from taking a look at it sooner. David did say that they have intentions to provide a formal base class, much like the code that he provided for me which will make creating these background hosted services easier. However, that won’t make it into the 2.0 release. The code I include in this sample might act as a good starting point until then, although it’s not fully tested.

Remember; there are no docs for this interface currently, so I’m taking a best guess at how it can be used and how it’s working based on what I’ve explored and been able to learn from David. This feature is preview and may also change before release (although very unlikely as 2.0 is nearly baked now). If you’re reading this in the future (and unless you’re a time traveller, you must be) please keep in mind this may be outdated.

Hosted Services

The first question to answer is what can we use this interface for? The basic idea is that it allows us to register background tasks, that run while our web host is running. These are co-ordinated with the lifetime of the application. We register a Task when the application starts and have the opportunity to do some graceful clean-up when the application is shutting down. While we could spin off work on a background thread previously, it would be killed when the main application process shutdown.

To create these background tasks we implement the new IHostedService interface.

The interface looks like this:

The idea is we register one or more implementations of this interface with the DI container, all of which will then be started and stopped along with the application by the HostedServiceExecutor. As users of this interface we are responsible for properly handling the cancellation and shutdown of our services when StopAsync is triggered by the host.

Creating a Hosted Service

One of the first possible use cases for these background tasks that I came up with was a scenario where we might want to update some content or data in our application from an external source, refreshing it periodically. Rather than doing that update in the main request thread, we can offload it to a background task.

In this simple example I provide an API endpoint which returns a random string value provided from an external service and updated every 5 seconds.

The first part of the code is a provider class which will hold the string value and which includes an update method that when called, will get a new string from the external service.

I then use this provider from my controller to return the string.

The main work for the setup of our service lives in an abstract base class called HostedService. This is the base class that David Fowler kindly put together. The code looks like this:

The comments in the class describe the flow. When using this base class we simply need to implement the ExecuteAsync method.

When the StartAsync method is called by the HostedServiceExecutor a CancellationTokenSource is created and linked to the token which was passed into the StartAsync method. This CancellationTokenSource is stored in a private field.

ExecuteAsync, an abstract method is then called, passing in the token from our CancellationTokenSource and the returned Task itself is stored. The StartAsync then must return as complete to the caller. A check is made in case our ExecuteAsync method is already completed. If not we return a Task.CompletedTask.

At this point we have a background task running whatever code we placed inside our implementation of ExecuteAsync. Our WebHost will go about its business of serving requests.

The other method defined by IHostedService is StopAsync. This method is called when the WebHost is shutting down. This is the key differentiator from running work on a traditional background thread. Since we have a proper hook into the shutdown of the host we can handle the proper shutdown of our workload on the background thread.

In the base class, first a check is made to ensure that StartAsync was previously called and that we actually have an executing task. Then we signal cancellation on our CancellationTokenSource. We await the completion of one of two things. The preferred option is that our Task which should be adhering to the cancellation token we passed to it, completes. The fall back is the Task.Delay(-1, cancellationToken) task completes. This takes in the cancellation token passed by the HostedServiceExecutor, which in turn is provided by the StopAsync method of the WebHost. By default this will by a token set with a 5 second timeout, although this timeout value can be configured when building our WebHost using the UseShutdownTimeout extension on the IWebHostBuilder. This means that our service is expected to cancel within 5 seconds otherwise it will be more abruptly killed.

To use this base class I then created a class inheriting from it called DataRefreshService. It is within this class that I implement the ExecuteAsync abstract method from HostedService. The ExcecuteAsync method accepts a cancellation token and looks like this:

I call the UpdateString method on the RandomStringProvider and then wait for 5 seconds before repeating. This all happens inside a while loop which continues indefinitely, until cancelation has been requested for the cancellation token. We pass the cancellation token down into the other async methods as well, so that they too can cancel their tasks all the way down the chain.

The final part of this is wiring up the dependency injection. Within the configure services method of Startup I must register the hosted service. I also register my RandomStringProvider class too, since that is passed into things via DI.

Summary

The IHostedService interface provides a nice way to properly start background work in a web application. It’s a feature we should not overuse, as I doubt it’s intended for spinning up large numbers of tasks, but for some scenarios it offers a nice solution. Its main benefit is the chance to perform proper cancellation and shutdown of our background tasks when the host itself is shutting down.

A special thanks to the amazing David Fowler for a quick code review (and correction) of my DataRefreshService. It was very kind of him to spare me some of his time to help me better understand this new feature. I hope I’ve explained everything correctly so that others can benefit from what he shared with me.

If you would like to view the source code from this post you can find it on my GitHub account.