HttpClientFactory in ASP.NET Core 2.1 (Part 4) Integrating with Polly for transient fault handling

In the previous post in this series, I introduced the concept of outgoing middleware using DelegatingHandlers registered with named and typed clients. While that approach is available, the ASP.NET team hope that for most scenarios, we won’t need to resort to manually building our own handlers. In some cases, the built-in features of the library may provide the functionality we need. For example, it is sometimes useful to wrap requests within timing code to track how long they take to execute. This is now built into IHttpClientFactory as part of its default logging. In other cases, third-party integration may provide the functionality you require. For example, a common cross-cutting concern is handling transient faults during HTTP requests. In this case, rather than crafting our own retry logic, it’s much better to use a library such as Polly.

Polly is a popular transient fault handling library which provides a mechanism to define policies which can be applied when certain failures occur. One of the more commonly used policies is the retry policy. This allows you to wrap some code which, should a failure occur, will be retried; multiple times if necessary. This is very useful in situations where your application needs to communicate with external services. There is the ever-present risk when communicating with services over a transport such as HTTP that a transient fault will occur. A transient fault may prevent your request from being completed but is also likely to be a temporary problem. This makes retrying a sensible option in those cases.

As well as retries, Polly offers a number of other types of policy, many of which you may want to combine with retry to build up sophisticated ways to deal with failures. I will cover a few of the more general examples in this post, but if you want more comprehensive coverage I recommend you check out the Polly wiki.

The ASP.NET team have worked closely with Dylan and Joel, the primary maintainers of Polly, to include an integration pattern to make applying Polly policies to HttpClient instances really straightforward.

Before we can work with the Polly integrations we need to add a package reference to our project. The general IHttpClientFactory functionality lives inside the Microsoft.Extensions.Http package which is included as a dependency in the Microsoft.AspNetCore.App 2.1 meta package. This is a new meta package in ASP.NET Core 2.1 which doesn’t include third-party dependencies. Therefore, in order to use the Polly extensions for IHttpClientFactory we need to add the Microsoft.Extensions.Http.Polly package to our project.

After doing so in a basic project the csproj file will look something like this:

Applying a Policy

The Microsoft.Extensions.Http.Polly package includes an extension method called AddPolicyHandler on the IHttpClientBuilder that we can use to add a handler which will wrap all requests made using an instance of that client in a Polly policy. The IHttpClientBuilder is returned when we define a named or typed client.

We can then use the extensions in our ConfigureServices method…

In this example, we’re defining a client named “github” and we’ve used the AddPolicyHandler method to pass in a timeout policy. The policy you provide here must be an IAsyncPolicy<HttpResponseMessage>. This policy will timeout any requests after 10 seconds.

Reusing Policies

When using Polly, where possible, it is a good practice to define policies once and share them in cases where the same policy should be applied. This way, to change the rules for a policy, those changes only need to be made in one place. Also, it ensures that the policy is allocated only once. Certainly, policies such as the circuit breaker need to be shared if multiple callers expect to run through the same circuit breaker instance. 

For this example, we’ll declare the timeout policy from the last example once and share it with two named clients…

We’ll look at another option for policy reuse a little later in this post when we explore using a PolicyRegistry.

Transient Fault Handling

When dealing with HTTP requests, the most common scenarios we want to handle are transient faults. As this is a common requirement, the Microsoft.Extensions.Http.Polly package includes a specific extension that we can use to quickly setup policies that handle transient faults.

For example, to add a basic retry when a transient fault occurs for requests from a named client we can register the retry policy as follows:

In this case, all requests made through the client will retry when certain failure conditions are met. The AddTransientHttpErrorPolicy method takes a Func<PolicyBuilder<HttpResponseMessage>, IAsyncPolicy<HttpResponseMessage>>. The PolicyBuilder here will be preconfigured to handle HttpRequestExceptions, any responses returning a 5xx status code and also any responses with a 408 (request timeout) status code. This should be suitable for many situations. If you require the policy to apply under other conditions, you will need to use a different overload to pass in a more specific policy.

Be aware; when performing retries we need to consider idempotency. Retrying a HTTP GET is a pretty safe operation. If we’ve made the call and not received any response, we can safely retry the call without any danger. However, consider what might happen if we retry a HTTP POST request. In that case, we have to be more careful since it’s possible that your original request was actually received, but the response we received suggested a failure. In that case, retrying could lead to duplication of data, or corruption of the data stored in the downstream system. Here, you need to have more knowledge of what the downstream service will do if it receives the same request more than once. Is retrying a safe operation? When you own the downstream service, it is easier to control this. You might, for example, use some unique identifier to prevent duplicate POSTs.

When you have less control of the downstream system or you know that a duplicate POST might have negative consequences, you will need to control your policy more carefully. An option that might be suitable is to define different named/typed clients. You could create one for those requests that have no side effects and another for those that do. You can then use the correct client for the action being taken. However, this might become a little difficult to manage. A better option is to use an overload of AddPolicyHandler which gives us access to the HttpRequestMessage so that policies can be applied conditionally. That overload looks like this:

AddPolicyHandler(Func<HttpRequestMessage, IAsyncPolicy<HttpResponseMessage>> policySelector)

You’ll note that the policySelector delegate here has access to the HttpRequestMessage and is expected to return an IAsyncPolicy<HttpResponseMessage>. We don’t have access to a PolicyBuilder setup to handle transient faults as we did in our earlier example. If we want to handle the common transient errors, we’ll need to define the expected conditions for our policy. To make this easier, the Polly project includes a helper extension that we can use that sets up a PolicyBuilder ready to handle the common transient errors. To use the extension method we need to add the Polly.Extensions.Http package from Nuget.

We can then call HttpPolicyExtensions.HandleTranisentHttpError() to get a PolicyBuilder that is configured with the transient fault conditions. We can use that PolicyBuilder to create a suitable retry policy which can then be conditionally applied when the request is a HTTP GET. In this example, any other HTTP methods use the NoOp policy.

Using a PolicyRegistry

The last example I want to cover in this post is a basic demonstration of how policies can be applied from a policy registry. To support policy reuse, Polly provides the concept of a PolicyRegistry which is essentially a container for policies. These can be defined at application startup by adding policies into the registry. The registry can then be passed around and used to access the policies by name.

The extensions available on the IHttpClientBuilder also support adding Polly based handlers to a client using a registry.

First, we must register a PolicyRegistry with DI. The Microsoft.Extensions.Http.Polly package includes some extension methods to make this simple. In the above example, I call the AddPolicyRegistry method which is an extension on the IServiceCollection. This will create a new PolicyRegistry and add register it in DI as the implementation for IPolicyRegistry<string> and IReadOnlyPolicyRegistry<string>. The method returns the policy so that we have access to add policies to it.

In this example, we’ve added two timeout policies and given them names. Now when registering a client we can call the AddPolicyHandlerFromRegistry method available on the IHttpClientBuilder. This takes the name of the policy we want to use. When the factory creates instances of this named client, it will add the appropriate handler, wrapping calls in the “regular” retry policy which will be retrieved from the registry.

Summary

As a long time user of Polly, I’m very happy to see the integration being added with IHttpClientFactory. Together these libraries make it really easy to get up and running with HttpClient instances that are able to handle transient faults seamlessly. The examples I’ve shown are quite basic and general, but I hope they give the idea of how policies can be used and registered. For more detailed Polly documentation and examples, I recommend you check out the Polly wiki. It was great being involved in some of the early discussions with both the ASP.NET and Polly teams when this integration was being designed as I was able to suggest the usefulness of the policy registry extensions.

Other Posts in this Series

Part 1 – An introduction to HttpClientFactory
Part 2 – Defining Named and Typed Clients
Part 3 – Outgoing request middleware with handlers
Part 4 – This post

HttpClientFactory in ASP.NET Core 2.1 (Part 3) Outgoing request middleware with handlers.

In my previous posts in this series (An Introduction to HttpClientFactory and Defining Named and Typed Clients) I introduced some core concepts and then showed some examples of using the new IHttpClientFactory feature in ASP.NET Core 2.1. It’s been a while since those first two posts but I’d like to continue this series by looking at the concept of outgoing request middleware with handlers.

IMPORTANT NOTE: the features shown here require the current preview build of the SDK and the .NET Core and ASP.NET Core libraries. I won’t cover how to get those in this post. At the time of writing we’re in preview 2 of .NET Core and ASP.NET Core 2.1. This preview should be reasonably feature complete but things may still change. If you want to try this out today you can get the preview 2 installers but I recommend waiting until at least the RC before producing any production code.

DelegatingHandlers

To be clear from the outset; many of the pieces involved in this part of the feature have existed for a long time. HttpClientFactory simply makes the consumption of these building blocks easier, through a more composable and clear API.

When making HTTP requests, there are often cross cutting concerns that you may want to apply to all requests through a given HttpClient. This includes things such as handling errors by retrying failed requests, logging diagnostic information or perhaps implementing a caching layer to reduce the number of HTTP calls on heavily used flows.

For those familiar with ASP.NET Core, you will also likely be familiar with the middleware concept. DelegatingHandlers offer an almost identical concept but in reverse; when making outgoing requests.

You can define a chain of handlers as a pipeline, which will all have the chance to process an outgoing HTTP request before it is sent. These handlers may choose to modify headers programmatically, inspect the body of the request or perhaps log some information about the request.

The HttpRequestMessage flows through each handler in turn under it reaches the final inner handler. This handler is what will actually dispatch the HTTP request across the wire. This inner handler will also be the first to receive the response. At this point that response passes back through the pieline of handlers in the reverse order. Again, each handler can inspect, modify or use the response as necessary. Perhaps for certain request paths you want to apply caching of the returned data for example.

 

IHttpClientFactory - DelegatingHandler outgoing middleware pipeline flow

In the diagram above you can see this pipeline visualised.

 

Much like ASP.NET Core middleware, it also possible for a handler to short-circuit the flow and return a response immediately. One example where this might be useful is to enforce certain rules you may have in place. For example, you could create a handler which checks if an API key header is present on outgoing requests. If this is missing, then it doesn’t pass the request along to the next handler (avoiding an actual HTTP call) and instead generates a failure response which it returns to the caller.

Before IHttpClientFactory and its extensions you would need to manually pass a handler instance (or chain of handlers) into the constructor for your HttpClient instance. That instance will then process any outgoing requests through the handlers it has been supplied.

With IHttpClientFactory we can more quickly apply one or more handlers by defining them when registering our named or typed clients. Now, anytime we get an instance of that named or typed client from the HttpClientFactory, it will be configured with the required handlers. The easier way to show this is with some code.

Creating a Handler

We’ll start by defining two handlers. In order to keep this code simple these aren’t going to be particularly realistic in terms of function. They will however show the key concepts. As we’ll see in future posts, there are ways to achieve similar results without having to write our own handlers.

To create a handler we can simply create a class which inherits from the DelegatingHandler abstract class. We can the override the SendAsync method to add our own functionality.

In our example this will be our outer request. A StopWatch will be started before calling and awaiting the base handler’s SendAsync method which will return a HttpResponseMessage. At this point the external request has completed. We can log out the total time taken for the request to flow through any other handlers, out to the endpoint over HTTP and for the response to be received.

Just to keep things interesting let’s create a second handler. This one will check for the existence of a header and if it is missing, will return an immediate response, short-circuiting the handler pipeline and avoiding an unnecessary HTTP call.

Registering Handlers

Now that we have created the handlers we wish to use, the final step is to register them with the dependency injection container and define a client. We perform this work in the ConfigureServices method of the Startup class.

The first two lines register each handler with the service collection which will be used to build the final service provider. These need to be transient so that a new instance is provided each time a new HttpClient is created.

Next, we define a client. In this example I’m using a named client for simplicity. Check out my previous post in this series for more detail about named and typed clients. The AddHttpClient method in this case returns an IHttpClientBuilder. We can call additional extension methods on this builder. Here we are calling the generic AddHttpMessageHandler method. This method takes the type for the handler as its generic parameter.

The order of registration matters here. We start by registering the outer most handler. This handler will be the first to inspect the request and the last to see the response. In this case we want our timing handler to record complete time taken for the whole request flow, including time spent in any inner handlers, so we have added it first. We can call AddHttpMessageHandler again, this time with our ValidateHeaderHandler handler. This will be our final custom handler before the inner HttpClientHandler is passed the request to send the request over the network.

At this point we now have an outgoing middleware pipeline defined on our named ‘github’ client. When a request comes through this client it will first pass into the TimingHandler, then into the ValidateHeaderHandler. Assuming the header is found the request it will be passed on and sent out to the URI in the request. When the response comes back it first returns through the ValidateHeaderHandler which does nothing with the response. It then passes onto the TimingHandler where the total elapsed time is logged and then finally is returned to the calling code.

Summary

While I have shown how easy it is to create a DelegatingHandler and then add it to your HttpClient outgoing pipeline using the new extensions; the team hope that for most cases you will not find yourself needing to craft your own handlers. Common concerns such as logging are taken care of for you within IHttpClientFactory (we’ll look at logging in a future post). For more complex but common requirements such as retrying failed requests and caching responses a much better option is to use a third party library called Polly. The team at Microsoft have made a great decision integrate with Polly.

In my next post I’ll investigate the options for adding Polly based handlers with IHttpClientFactory. In the mean time I suggest you check out this post by Scott Hanselman where he covers the Polly extensions. You can also check out the Polly wiki for more information.

Other Posts in this Series

Part 1 – An introduction to HttpClientFactory
Part 2 – Defining Named and Typed Clients
Part 3 – This post
Part 4 – Integrating with Polly for transient fault handling