HttpClientFactory in ASP.NET Core 2.1 (Part 1) An Introduction to HttpClientFactory

TL;DR;

A new HttpClientFactory feature is coming in ASP.NET Core 2.1 which helps to solve some common problems that developers may run into when using HttpClient instances to make external web requests from their applications.

Introduction

This blog post has been in the works since mid-October 2017, which was when I first noticed the new HttpClientFactory repository appear on GitHub. I was intrigued by its appearance and wondered what the ASP.NET team were up to, so I went diving into the available code that the repo contained at the time. I’ve then kept an eye on it ever since, watching as the team evolved the feature by reading the commits, issues and pull request discussions.

Recently the feature has started to be talked about more openly and was included in a recent talk by Damian Edwards and David Fowler at NDC London. In fact on the day of writing this introduction it’s been shown on both Jeff Fritz’s livestream show and the ASP.NET Community Standup. The opinion of Ryan Nowak, one of the main ASP.NET developers for the feature, is that it’s reasonably stable to begin writing about it now.

NOTE: Please bear in mind that this post is written prior to the official preview release of .NET Core 2.1 by using the nightly builds of ASP.NET Core 2.1 and the .NET Core SDK. Therefore, things may change before and during the public previews (hopefully we’ll get these within the next month) and also before the final release of 2.1 based on feedback received from those previews.

What is HttpClientFactory?

In the words of the ASP.NET Team it is “an opinionated factory for creating HttpClient instances” and is a new feature coming with the release of ASP.NET Core 2.1. Depending on your past experience using HttpClient, you may or may not be aware of some of the pitfalls that can be encountered, sometimes without even being aware that you have a problem.

The first issue is when you create too many HttpClients within your code which can in turn create two problems…

  1. It’s inefficient as each one will have its own connection pool for the remote server. This means you pay the cost of reconnecting to that remote server for every client you create.
  2. The bigger problem you can have if you create a lot of them is that you can run into socket exhaustion where you have basically used up too many sockets too fast. There is a limit on how many sockets you can have open at one time. When you dispose of the HttpClient, the connection it had open remains open for up to 240 seconds in a TIME_WAIT state (in case any packets from the remote server still come through).

HttpClient implements IDisposable and this often leads developers to follow the normal pattern when using an IDisposable object, creating it within a using block. This ensures that the object is properly disposed of once you’re done with it and it has gone out of scope. If you want to read more about this, it is well documented by the ASP.NET Monsters in their post “You’re using HttpClient wrong and it’s destablizing your software”.

A preferred approach therefore it to reuse HttpClient instances so that connections can also be reused. HttpClient is a mutable object but as long as you are not mutating it, it is actually thread safe and can be shared. A common approach is therefore to register it as a singleton with a DI framework or to create a wrapper around it which holds a static instance.

However, this creates a new problem. Using a single HttpClient in this way will keep connections open and not respect the DNS Time To Live (TTL) setting. Now the connections will never get DNS updates so the server you are talking to will never have its address updated. This is entirely possible in some situations where you are balancing over many hosts that may go away over time or perhaps rolling out new services using blue/green deployments. If the server is gone, the IP your connection is using may no longer respond to requests that you make through the single HttpClient. You can read more about this issue at “Singleton HttpClient? Beware of this serious behaviour and how to fix it” and “Singleton HttpClient doesn’t respect DNS changes”.

HttpClientFactory is designed to help start solving these problems and provides a new mechanism to create HttpClient instances that are properly managed for us behind the scenes. It will “do the right thing” for us and we can focus on other things! While the above problems are mentioned in reference to HttpClient, in fact the source of the issues actually occurs on the HttpClientHandler, which is used by HttpClient. The HttpClientFactory manages the lifetime of the handlers so that we have a pool of them which can be reused, while also rotating them so that DNS doesn’t get stale.

The expensive part of using HttpClient is actually creating the HttpClientHandler and the connection. Having these pooled in this manner means we can get more efficient use of the connections on our system. When you use the HttpClientFactory to request a HttpClient, you do in fact get a new instance each time, which means we don’t have to worry about mutating it’s state. This HttpClient may (or may not) use an existing HttpClientHandler from the pool and therefore use an existing open connection.

By default, each new HttpClientHandler (which derives from HttpMessageHandler) will be created with an active lifetime of 2 minutes. This can be controlled on a per named client basis when creating it’s handler chain. Once the lifetime is reached, the handler will not be immediately be disposed of and will instead be placed into the expired pool. Any clients depending on the original handler chain can continue using it without any issues. There is a background job checking the expired pool to see if all references for the handler have gone out of scope, at which point it can then be disposed of. Any new requests for a new client once the handler chain has been expired will get a new handler chain.

This works reasonably well, but there are other things underway on the .NET Core side which might improve the situation further. The .NET Core team are working on a new ManagedHandler which should manage DNS more correctly and in principle can be kept around for longer, meaning connections can be shared even more efficiently. This new handler is also being designed to function more consistently across the different operating systems. Until that work is completed (which might be in the 2.1 time frame) the pooling of handlers above is a reasonable workaround.

How to use HttpClientFactory

IMPORTANT NOTE: The features and code samples shown here require the current nightly builds of the SDK and the .NET Core and ASP.NET Core libraries. I won’t cover how to get setup to use those in this post. Treat this as an early preview of how the feature will work so that you can begin planning why, where and how you will use it once 2.1 is publicly available. Unless you have an urgent need to try this out today, I’d recommend waiting until the 2.1 previews are released, hopefully within the next month or so.

In this post I’ll concentrate on one of the most basic ways to get started with the HttpClientFactory. For this example, we’ll start by creating a simple WebAPI project and then edit the csproj file to upgrade it to use the new .NET Core and ASP.NET Core 2.1 bits. First we need to set it to be based on netcoreapp2.1 (not yet in official preview) and then include two packages which we’ll need. For this post I’m pinning those to specific preview nightly build versions available on the ‘dev’ MyGet feeds. After doing this our project file looks like this:

Next we need to head over to our Startup.cs file and register a service. The HttpClientFactory includes various ServiceCollection extensions. The one we’ll use for this example is:

services.AddHttpClient();

Behind the scenes this will register a few required services, one of which will be an implementation of IHttpClientFactory. Next we’ll update the default ValuesController to make use of this feature:

Here we are first adding a dependency on IHttpClientFactory which will be injected into our controller by the DI system. The IHttpClientFactory allows us to ask for and receive a HttpClient instance.

Within our Get action we are then using the HttpClientFactory to create a client. Behind the scenes the HttpClientFactory will create a new HttpClient for us. But wait, didn’t I say earlier that using a new HttpClient for each request is bad? Indeed I did; but in fact that was a little bit of misdirection. The HttpClient itself is not really the problem, it’s the HttpClientHandler which it uses to make the HTTP calls that is the actual issue. It’s this which opens the connections to the external services that will then remain open and block sockets, even in the main HttpClient is disposed of.

HttpClientFactory pools these HttpClientHandler instances and manages their lifetime in order to solve some of the issues I mentioned earlier. Each time we ask for a HttpClient, we get a new instance, which may (or may not) use an existing HttpClientHandler. The HttpClient itself it not too heavy to construct so this is okay.

Once created the HttpClientHandlers are pooled and held around for around 2 minutes by default. This means that any new requests for CreateClient may share a handler and therefore the connections also. While a HttpClient lives, it’s handler will remain available and again this will share the connection.

After the two minutes, each HttpClientHandler is marked as expired. The expired state simply marks them so that they are no longer used when creating any new HttpClient instances. They are not immediately disposed however, as other HttpClient instances may be using them. The HttpClientFactory uses a background service which monitors the expired handlers and once they are no longer referenced, can then dispose of them properly, allowing their connections to be closed also.

This pooling feature helps reduce the risk of socket exhaustion and the refreshing process helps solve the DNS update problem by ensuring we don’t have long lived instances of HttpClientHandlers and connections hanging around. It’s a reasonable compromise which is managed for us by making use of the HttpClientFactory feature.

Summary

I’ll leave it there for this introductory post. In future posts I’ll dive into some of the more advanced ways we can use HttpClientFactory as there’s some nice features so show off. We’ll look at how we can create named HttpClient instances with configuration and also creating our own typed clients. This is where the feature will really begin to shine. Hopefully you’ll have seen, even in this basic example, how it improves use cases where you have a requirement to make HTTP calls in the most correct and efficient way. We don’t need to think about how we manage the lifetime of the clients or worry about running into DNS issues. I’m looking forward to using this in production once ASP.NET Core 2.1 is released.

Other Posts in this Series

Part 1 – This post
Part 2 – Defining Named and Typed Clients

  • Simon Timms

    Great post Steve. I’m glad to see that this rather messy problem with HTTPClient is now being properly addressed. It is probably a worthwhile exercise to create a Roslyn analyser to catch cases where people are manually creating instances of HTTP Client.

    • Steve Gordon

      Thanks Simon. Agree that it’s nice to have a formal solution that clear up the usage. I bet there’s a lot of code out there that should probably consider using this once released. I like the Roslyn analyser idea.

  • Interesting how this new pattern is similar to what I built in 2016 (which later blogged about: http://www.nimaara.com/2016/11/01/beware-of-the-net-httpclient/)

    The IRestClient (API compatible with HttpClient) can be injected into your classes it also addresses the DNS issues.

    It’s a shame that MSFT hasn’t made available an interface e.g. IHttpClient instead of (or in addition to) the HttpClientFactory.

    • Stef Heyenrath

      The link is incorrect.

  • As stated in that github issue, DNS updates have nothing to do with HTTP connections. A connection (which by default uses `keep-alive`) will stay alive until either end decides to close it. This is how it’s supposed to work and is 100% within spec.

    DNS is only used to lookup a name when making a connection in the first place, and that’s only if using a name instead of an IP directly. If the HTTP connection isn’t broken, there’s no reason to lookup the DNS again just because it has a shorter TTL than your connection has been alive.

    • James Gregory

      Azure use DNS changes to perform deploys from e.g. staging -> prod, so keeping connections open forever doesn’t really work there, and obviously lots of .NET developers use Azure. I’m not even sure what most Azure developers are doing to solve this at the moment. Using machines with stable ip addresses in AWS it isn’t such an issue.

      • Right, that’s a situation where the protocols are working as designed so it’s up to the implementation to handle it. The connections to the older version of the app should be broken at some point as Azure winds it down, or the clients should disconnect if you have control over them.

        If there’s a load balancer in front, then usually that will handle reconnecting to the correct/newer backend server while the client doesn’t know about or notice any connection breaks.

  • toby hei

    Nice series! It would be great to be able to use this beyond asp net core. Do you happen to know whether there are any plans to release this as a stand-alone package or as a Microsoft extensions package rather than an asp net core one?

    • Steve Gordon

      Thanks Toby. Glad you’ve found it useful so far. It’s a net standard package so you can use it in other .NET Core apps such as a console app for example. However, it does somewhat depend on the Microsoft DI system. There’s a new HostBuilder coming in .Net 2.1 which will make setting that sort of stuff up a bit easier, although you can do it manually too. I can’t see this easily working with .NET Framework though but I haven’t investigated if there may be workarounds.

  • Chet

    Awesome. This would be so helpfully when utilizing/ creating .net core Web APi and consuming it! Thanks for sharing, can’t wait for it!