Working with Polly – Using the Context to Obtain the Retry Count for Diagnostics Exploring a basic use case the for Polly Context object to track diagnostic data

I’ve been using Polly for a number of years now. For the most part, my usage of the library has been to solve some quite basic problems; commonly to implement retry logic when calling external services for example. In this post, I want to explore a requirement I had when using Polly within a library that would be shared with various other internal projects.

The library acts as an “SDK” of sorts, wrapping an API we have written. This enables our consuming services to reference the package if they need to consume the API, avoiding repetition of the code required to interact with it.

The scenario that this post will focus on is how we can capture diagnostic information during policy execution for use in application monitoring.

The Requirement

Let’s take a simple example. We want to record application metrics about the number of retries that each attempt to call a third party service requires. I want to be able to get this information after the execution of the code that is wrapped in the policy is complete. In my case, some of my consuming applications will record this data to a third-party service called DataDog. We can then track and monitor this metric over time to understand if calls to the API are degraded.

I don’t want to force my SDK library to depend on DataDog directly since I can’t assume all consumers will need, nor want to record metrics. Instead, I want to capture data during policy execution which my SDK can pass back as part of a result object to the caller. Consuming projects can then choose to use that information if they need to, or disregard it if they don’t.

NOTE: One thing I should highlight at this point is that the Polly team are actively planning work on a diagnostics feature for Polly. Once that work is completed and becomes available then solving this requirement will become simpler.

The solution which I’ve come up with in the meantime is to utilise the Polly Context. Essentially the Context allows you to pass in a set of objects which can then be accessed during policy execution. The context includes a (lazily-initialised) dictionary to store any data/objects that you want.

The way I chose to implement my requirement was to set up the context and attach it to the policy executing the retry around the HTTP request.

Creating the policy

NOTE: Our library makes use of the new HttpClientFactory feature in .NET Core 2.1, so the examples here will mostly focus on that use case.

First, we’ll define a policy which will execute our HTTP request and utilise the Context to record the retries.

Update – 26-07-2018

Since publishing this post I’ve discussed this sample with Dylan Reisenberger who expertly suggested this can be simplified if we instead use the built-in retryCount to set the value on our context. 

We no longer need to increment our own count and can set/update the value for the “retrycount” key with less code.

Here we’ve used the HttpPolicyExtensions to help create a policy which will retry any transient errors that occur when making the request.

The WaitAndRetryAsync method, as one of its overloads, accepts an Action delegate, which as one of its arguments includes the Context object. In the preceding example, I try to access an item with the key “retrycount” from the Context dictionary. Using pattern matching we can check that the value is an integer and if so, assign it to a local variable called retries.

In the case where this value is available, we can then increment the retries value and assign it back into the context against the retrycount key.

Executing the policy

Before executing code wrapped in the policy, we need to create a Context to pass to the execution. Creating a Context is as simple as allocating one and adding an entry in its internal dictionary.

In the preceding code, we’ve created a context and added a retrycount integer to it, initialised with a value of zero. This can then be used to track the number of attempts made during an execution of a retry based policy.

With a Context object created, we can go ahead and pass it into the policy execution. The standard way to do this is to pass it as an argument to the Execute or ExecuteAsync method when utilising the policy. For example:

However, in my case, I am using the new HttpClientFactory feature. When using HttpClientFactory, clients are defined in the ConfigureServices method with any required Polly policies being added using the various Polly extension methods on the IHttpClientBuilder. See my previous post for more detail of how to use Polly with IHttpClientFactory.

With HttpClientFactory, we don’t directly execute the policy. That is done for us within the handlers.

To support the use of the context with HttpClientFactory, an extension method on the HttpRequestMessage is provided called SetPolicyExecutionContext. This accepts a Polly Context object which it then adds it to the request properties (a Dictionary<string, object>). During execution, the handler can access the context from the request and pass it into the policy it is executing.

We’ll use that approach in this case. Firstly we create the request and then call the SetPolicyExecutionContext to apply our context object:

We can then get a client from the HttpClientFactory. There are various ways to achieve this which I’ve covered in my HttpClientFactory series. For this example we manually use the factory to create a fresh client:

This client has had the retry policy added to it when defining it in the ConfigureServices method…

After the policy has executed, the retrycount can be accessed from the original reference to context object that we attached to the request.

At this point, I can add the retryCount value to an object which my library passes back to the caller. The caller can use that information if it needs to in order to log events or record DataDog metrics. I won’t include that code here.


Hopefully, this post demonstrates how easy it is to use the Polly context to pass data into and back out of the execution of policies. This is proving useful for my current scenarios as it allows general policies to be defined centrally which can then be used in multiple places. Remember, in future versions of Polly we can expect some new diagnostic functionality, perhaps in the form of events, which we can hook into to give a richer insight into details such as the number of executed retry attempts. For now, this quite straightforward approach is a solution which I’m pretty happy with.

HttpClientFactory in ASP.NET Core 2.1 (Part 4) Integrating with Polly for transient fault handling

In the previous post in this series, I introduced the concept of outgoing middleware using DelegatingHandlers registered with named and typed clients. While that approach is available, the ASP.NET team hope that for most scenarios, we won’t need to resort to manually building our own handlers. In some cases, the built-in features of the library may provide the functionality we need. For example, it is sometimes useful to wrap requests within timing code to track how long they take to execute. This is now built into IHttpClientFactory as part of its default logging. In other cases, third-party integration may provide the functionality you require. For example, a common cross-cutting concern is handling transient faults during HTTP requests. In this case, rather than crafting our own retry logic, it’s much better to use a library such as Polly.

Polly is a popular transient fault handling library which provides a mechanism to define policies which can be applied when certain failures occur. One of the more commonly used policies is the retry policy. This allows you to wrap some code which, should a failure occur, will be retried; multiple times if necessary. This is very useful in situations where your application needs to communicate with external services. There is the ever-present risk when communicating with services over a transport such as HTTP that a transient fault will occur. A transient fault may prevent your request from being completed but is also likely to be a temporary problem. This makes retrying a sensible option in those cases.

As well as retries, Polly offers a number of other types of policy, many of which you may want to combine with retry to build up sophisticated ways to deal with failures. I will cover a few of the more general examples in this post, but if you want more comprehensive coverage I recommend you check out the Polly wiki.

The ASP.NET team have worked closely with Dylan and Joel, the primary maintainers of Polly, to include an integration pattern to make applying Polly policies to HttpClient instances really straightforward.

Before we can work with the Polly integrations we need to add a package reference to our project. The general IHttpClientFactory functionality lives inside the Microsoft.Extensions.Http package which is included as a dependency in the Microsoft.AspNetCore.App 2.1 meta package. This is a new meta package in ASP.NET Core 2.1 which doesn’t include third-party dependencies. Therefore, in order to use the Polly extensions for IHttpClientFactory we need to add the Microsoft.Extensions.Http.Polly package to our project.

After doing so in a basic project the csproj file will look something like this:

Applying a Policy

The Microsoft.Extensions.Http.Polly package includes an extension method called AddPolicyHandler on the IHttpClientBuilder that we can use to add a handler which will wrap all requests made using an instance of that client in a Polly policy. The IHttpClientBuilder is returned when we define a named or typed client.

We can then use the extensions in our ConfigureServices method…

In this example, we’re defining a client named “github” and we’ve used the AddPolicyHandler method to pass in a timeout policy. The policy you provide here must be an IAsyncPolicy<HttpResponseMessage>. This policy will timeout any requests after 10 seconds.

Reusing Policies

When using Polly, where possible, it is a good practice to define policies once and share them in cases where the same policy should be applied. This way, to change the rules for a policy, those changes only need to be made in one place. Also, it ensures that the policy is allocated only once. Certainly, policies such as the circuit breaker need to be shared if multiple callers expect to run through the same circuit breaker instance. 

For this example, we’ll declare the timeout policy from the last example once and share it with two named clients…

We’ll look at another option for policy reuse a little later in this post when we explore using a PolicyRegistry.

Transient Fault Handling

When dealing with HTTP requests, the most common scenarios we want to handle are transient faults. As this is a common requirement, the Microsoft.Extensions.Http.Polly package includes a specific extension that we can use to quickly setup policies that handle transient faults.

For example, to add a basic retry when a transient fault occurs for requests from a named client we can register the retry policy as follows:

In this case, all requests made through the client will retry when certain failure conditions are met. The AddTransientHttpErrorPolicy method takes a Func<PolicyBuilder<HttpResponseMessage>, IAsyncPolicy<HttpResponseMessage>>. The PolicyBuilder here will be preconfigured to handle HttpRequestExceptions, any responses returning a 5xx status code and also any responses with a 408 (request timeout) status code. This should be suitable for many situations. If you require the policy to apply under other conditions, you will need to use a different overload to pass in a more specific policy.

Be aware; when performing retries we need to consider idempotency. Retrying a HTTP GET is a pretty safe operation. If we’ve made the call and not received any response, we can safely retry the call without any danger. However, consider what might happen if we retry a HTTP POST request. In that case, we have to be more careful since it’s possible that your original request was actually received, but the response we received suggested a failure. In that case, retrying could lead to duplication of data, or corruption of the data stored in the downstream system. Here, you need to have more knowledge of what the downstream service will do if it receives the same request more than once. Is retrying a safe operation? When you own the downstream service, it is easier to control this. You might, for example, use some unique identifier to prevent duplicate POSTs.

When you have less control of the downstream system or you know that a duplicate POST might have negative consequences, you will need to control your policy more carefully. An option that might be suitable is to define different named/typed clients. You could create one for those requests that have no side effects and another for those that do. You can then use the correct client for the action being taken. However, this might become a little difficult to manage. A better option is to use an overload of AddPolicyHandler which gives us access to the HttpRequestMessage so that policies can be applied conditionally. That overload looks like this:

AddPolicyHandler(Func<HttpRequestMessage, IAsyncPolicy<HttpResponseMessage>> policySelector)

You’ll note that the policySelector delegate here has access to the HttpRequestMessage and is expected to return an IAsyncPolicy<HttpResponseMessage>. We don’t have access to a PolicyBuilder setup to handle transient faults as we did in our earlier example. If we want to handle the common transient errors, we’ll need to define the expected conditions for our policy. To make this easier, the Polly project includes a helper extension that we can use that sets up a PolicyBuilder ready to handle the common transient errors. To use the extension method we need to add the Polly.Extensions.Http package from Nuget.

We can then call HttpPolicyExtensions.HandleTranisentHttpError() to get a PolicyBuilder that is configured with the transient fault conditions. We can use that PolicyBuilder to create a suitable retry policy which can then be conditionally applied when the request is a HTTP GET. In this example, any other HTTP methods use the NoOp policy.

Using a PolicyRegistry

The last example I want to cover in this post is a basic demonstration of how policies can be applied from a policy registry. To support policy reuse, Polly provides the concept of a PolicyRegistry which is essentially a container for policies. These can be defined at application startup by adding policies into the registry. The registry can then be passed around and used to access the policies by name.

The extensions available on the IHttpClientBuilder also support adding Polly based handlers to a client using a registry.

First, we must register a PolicyRegistry with DI. The Microsoft.Extensions.Http.Polly package includes some extension methods to make this simple. In the above example, I call the AddPolicyRegistry method which is an extension on the IServiceCollection. This will create a new PolicyRegistry and add register it in DI as the implementation for IPolicyRegistry<string> and IReadOnlyPolicyRegistry<string>. The method returns the policy so that we have access to add policies to it.

In this example, we’ve added two timeout policies and given them names. Now when registering a client we can call the AddPolicyHandlerFromRegistry method available on the IHttpClientBuilder. This takes the name of the policy we want to use. When the factory creates instances of this named client, it will add the appropriate handler, wrapping calls in the “regular” retry policy which will be retrieved from the registry.


As a long time user of Polly, I’m very happy to see the integration being added with IHttpClientFactory. Together these libraries make it really easy to get up and running with HttpClient instances that are able to handle transient faults seamlessly. The examples I’ve shown are quite basic and general, but I hope they give the idea of how policies can be used and registered. For more detailed Polly documentation and examples, I recommend you check out the Polly wiki. It was great being involved in some of the early discussions with both the ASP.NET and Polly teams when this integration was being designed as I was able to suggest the usefulness of the policy registry extensions.

Other Posts in this Series

Part 1 – An introduction to HttpClientFactory
Part 2 – Defining Named and Typed Clients
Part 3 – Outgoing request middleware with handlers
Part 4 – This post