ASP.NET Core 2.2 First Look – Endpoint Routing (aka Dispatcher) What is Endpoint Routing in ASP.NET Core 2.2?

The first preview of ASP.NET Core 2.2 is due (very) soon and it will be our first chance to test the changes and new features which are expected to be released by the end of this year.

ASP.NET Core 2.2 marks the next minor release of the ASP.NET Core framework, following on from the ASP.NET Core 2.1 release at the end of May. With these minor releases, the team are able to provide new features to us quickly and iterate on an already great product. ASP.NET Core 2.2 will be a “Current” release for those wanting to take advantage of the latest features more quickly. For some, the “current” releases may not be suitable as while offering newer features they do not offer the same long support commitment as long-term support (LTS). If you choose the “current” release train then you need to keep updating as new “current” releases are available (See the ASP.NET Core support policy for more details).

I’ve been watching the features planned for ASP.NET Core 2.2 since version 2.1 shipped. There are quite a few things which look pretty interesting. Today I want to take an early look at a new feature whose final name is Global Endpoint Routing. You may also see this referred to by its former name “Dispatcher”.

Update 18-08-2018: James Newton-King who works on the ASP.NET team, has confirmed that the name for this feature with be Endpoint routing.

NOTE: This post is based on reading and watching some limited early coverage of Dispatcher (Global Routing) from the ASP.NET team over the last few months. I’ve also spent a small amount of time looking at the source code and some early samples to form the content below. As a result, there are things I may not have 100% correct! This post is written prior to the first preview of ASP.NET Core 2.2 and as a result, some of this could change before the final release of ASP.NET Core 2.2. If you’re reading this in the future, treat it with some caution and keep an eye out for more recent posts where I will cover using the released versions of the feature. If you’re reading this in the past, I’m pleased to see that time travel is now a thing!!

Update 18-08-2018: After feedback from James Newton-King who works on the ASP.NET team, I understand that in the 2.2 timeframe the only supported way to use Endpoint Routing will be via MVC.

What is Global Endpoint Routing (the feature formerly known as Dispatcher)?

The current routing story with ASP.NET Core 2.1 MVC is that we define routes within MVC using either convention-based routing which uses a RouteBuilder when we register MVC into the middleware pipeline or via Route Attributes on our action methods. Personally, I tend towards the attribute approach for the APIs I build.

When a request reaches MVC, the route information is used to determine which Controller and Action should be invoked to handle the request, based on the URL path. At this point, MVC can invoke that action which will then return the response. If a suitable route is not found then a 404 is returned instead.

One issue with this approach to routing is that as the request flows through the middleware chain, all other middleware has no idea of where that request is eventually going to be handled.

With the Global Endpoint Routing feature, routing decisions can occur earlier into the pipeline so that incoming requests can be mapped to their eventual endpoint before MVC is even invoked. This gives other middleware an opportunity to make more reasoned choices and process requests armed with the knowledge of which Endpoint will eventually handle the request.

Implemented as middleware the URL matching can now be registered to run very early in the application middleware pipeline. After which, a set of known Endpoints will be registered. When a request is passed through the pipeline, other middleware will be able to inspect the final matched Endpoint and view/update its associated metadata.

In addition, Global Endpoint Routing looks like it should allow developers to build ASP.NET Core applications that don’t use MVC, but which can still rely on routing to handle requests. Developers can provide their own EndpointDataSource, complete with a set of route patterns and the corresponding RequestDelegates to be invoked for matches. There are opportunities here for some very lightweight APIs to be built without the sometimes unnecessary additional overhead of MVC.

Note: Based on the feedback from James Newton-King this feature is expected to work with MVC as the support way to use it in 2.2. For the 3.0 timeframe a public API for non MVC use is expected to be ready.

The original ASP.NET Core 2.2 features announcement (where the feature is known as “Dispatcher”) included a good example of where this Global Endpoint Routing concept can be useful. CORS is currently implemented as both a feature of MVC and as middleware. This is necessary as some requests will be handled outside of MVC, such as serving static files. However, to apply CORS for requests which do reach MVC, the policy can’t be applied until the MVC routing selects the controller and action for the request. Global Routing will enable CORS to be implemented more consistently since the routing and Endpoints will be known up front when a request is matched through the middleware.

Global Endpoint Routing is also integrated with the latest ASP.NET Core 2.2 MVC functionality, allowing MVC to work on top of this new Global Endpoint Routing feature. MVC in ASP.NET Core 2.2 includes code changes to support building up a list of available Endpoints for Global Endpoint Routing, using the existing MVC route provider.

I plan to dive into all of this more deeply in future posts. I’m currently spending a fair bit of time in the Microsoft.AspNetCore.Routing source code understanding the building blocks and Types involved.

A Small Sample

I’ll end with a basic example of what the services and middleware pipeline for an ASP.NET Core 2.2 application using Global Endpoint Routing could look like. In this case, there’s no MVC involved (this is based on the current RoutingSample from the Routing repository).

Update 18-08-2018: After feedback from James Newton-King who works on the ASP.NET team, I understand that in the 2.2 timeframe the only supported way to use Endpoint Routing will be via MVC.

Inside the Startup class, we must register the routing services in the ConfigureServices method as follows…

This sample manually adds a DefaultEndpointDataSource with a single Endpoint which is not necessary if you wish to use the MVC routes. MVC will populate Endpoints for you.

Once registered we can add Global Endpoint Routing to the pipeline as middleware in the Configure method…

The first middleware, UseEndpointRouting is responsible for inspecting the incoming request and matching it to an Endpoint. The matched Endpoint is set in the HttpContext.Features collection. At this point, all other middleware can retrieve the Endpoint from the Features collection as necessary.

Since we’re not using MVC in this sample we need something to invoke the matched RequestDelegate for the Endpoint. UseEndpoint registers the middleware which does exactly that.

Summary

Hopefully, this has been a useful primer of what to expect with the Global Endpoint Routing feature in ASP.NET Core 2.2. I expect this to be available in the first preview of ASP.NET Core 2.2 (perhaps due by the end of the month). I have started building up some fairly extensive notes as I explore the code and I hope to put together some follow up posts demonstrating more concrete samples once the previews are formally available (and at which point the API surface is less likely to change). I may also produce a deeper dive into how this all functions internally as its quite interesting code to explore.

Working with Polly – Using the Context to Obtain the Retry Count for Diagnostics Exploring a basic use case the for Polly Context object to track diagnostic data

I’ve been using Polly for a number of years now. For the most part, my usage of the library has been to solve some quite basic problems; commonly to implement retry logic when calling external services for example. In this post, I want to explore a requirement I had when using Polly within a library that would be shared with various other internal projects.

The library acts as an “SDK” of sorts, wrapping an API we have written. This enables our consuming services to reference the package if they need to consume the API, avoiding repetition of the code required to interact with it.

The scenario that this post will focus on is how we can capture diagnostic information during policy execution for use in application monitoring.

The Requirement

Let’s take a simple example. We want to record application metrics about the number of retries that each attempt to call a third party service requires. I want to be able to get this information after the execution of the code that is wrapped in the policy is complete. In my case, some of my consuming applications will record this data to a third-party service called DataDog. We can then track and monitor this metric over time to understand if calls to the API are degraded.

I don’t want to force my SDK library to depend on DataDog directly since I can’t assume all consumers will need, nor want to record metrics. Instead, I want to capture data during policy execution which my SDK can pass back as part of a result object to the caller. Consuming projects can then choose to use that information if they need to, or disregard it if they don’t.

NOTE: One thing I should highlight at this point is that the Polly team are actively planning work on a diagnostics feature for Polly. Once that work is completed and becomes available then solving this requirement will become simpler.

The solution which I’ve come up with in the meantime is to utilise the Polly Context. Essentially the Context allows you to pass in a set of objects which can then be accessed during policy execution. The context includes a (lazily-initialised) dictionary to store any data/objects that you want.

The way I chose to implement my requirement was to set up the context and attach it to the policy executing the retry around the HTTP request.

Creating the policy

NOTE: Our library makes use of the new HttpClientFactory feature in .NET Core 2.1, so the examples here will mostly focus on that use case.

First, we’ll define a policy which will execute our HTTP request and utilise the Context to record the retries.


Update – 26-07-2018

Since publishing this post I’ve discussed this sample with Dylan Reisenberger who expertly suggested this can be simplified if we instead use the built-in retryCount to set the value on our context. 

We no longer need to increment our own count and can set/update the value for the “retrycount” key with less code.


Here we’ve used the HttpPolicyExtensions to help create a policy which will retry any transient errors that occur when making the request.

The WaitAndRetryAsync method, as one of its overloads, accepts an Action delegate, which as one of its arguments includes the Context object. In the preceding example, I try to access an item with the key “retrycount” from the Context dictionary. Using pattern matching we can check that the value is an integer and if so, assign it to a local variable called retries.

In the case where this value is available, we can then increment the retries value and assign it back into the context against the retrycount key.

Executing the policy

Before executing code wrapped in the policy, we need to create a Context to pass to the execution. Creating a Context is as simple as allocating one and adding an entry in its internal dictionary.

In the preceding code, we’ve created a context and added a retrycount integer to it, initialised with a value of zero. This can then be used to track the number of attempts made during an execution of a retry based policy.

With a Context object created, we can go ahead and pass it into the policy execution. The standard way to do this is to pass it as an argument to the Execute or ExecuteAsync method when utilising the policy. For example:

However, in my case, I am using the new HttpClientFactory feature. When using HttpClientFactory, clients are defined in the ConfigureServices method with any required Polly policies being added using the various Polly extension methods on the IHttpClientBuilder. See my previous post for more detail of how to use Polly with IHttpClientFactory.

With HttpClientFactory, we don’t directly execute the policy. That is done for us within the handlers.

To support the use of the context with HttpClientFactory, an extension method on the HttpRequestMessage is provided called SetPolicyExecutionContext. This accepts a Polly Context object which it then adds it to the request properties (a Dictionary<string, object>). During execution, the handler can access the context from the request and pass it into the policy it is executing.

We’ll use that approach in this case. Firstly we create the request and then call the SetPolicyExecutionContext to apply our context object:

We can then get a client from the HttpClientFactory. There are various ways to achieve this which I’ve covered in my HttpClientFactory series. For this example we manually use the factory to create a fresh client:

This client has had the retry policy added to it when defining it in the ConfigureServices method…

After the policy has executed, the retrycount can be accessed from the original reference to context object that we attached to the request.

At this point, I can add the retryCount value to an object which my library passes back to the caller. The caller can use that information if it needs to in order to log events or record DataDog metrics. I won’t include that code here.

Summary

Hopefully, this post demonstrates how easy it is to use the Polly context to pass data into and back out of the execution of policies. This is proving useful for my current scenarios as it allows general policies to be defined centrally which can then be used in multiple places. Remember, in future versions of Polly we can expect some new diagnostic functionality, perhaps in the form of events, which we can hook into to give a richer insight into details such as the number of executed retry attempts. For now, this quite straightforward approach is a solution which I’m pretty happy with.

HttpClientFactory in ASP.NET Core 2.1 (Part 5): Logging Exploring the default request and response logging and how to replace the logging implementation

In the 2.1 release of IHttpClientFactory, the ASP.NET team included some built-in logging of the HTTP calls made via HttpClients created by the factory. This can be useful for the diagnosis of failures, as well as to understand the time taken to complete HTTP calls to external services. 

In this post, I want to explore what is available in the default logging, how we can control what gets logged, how the logging is implemented and finally, how we can replace the logging with our implementation.

There’s quite a bit of technical detail in this post. I hope it proves useful and interesting for those working with the IHttpClientFactory. Click here, if you want to jump to the section on customising the logging by replacing the default implementation.

What’s in the logs?

IHttpClientFactory includes two levels of logging. At information level, the time taken to process and send the request is included. This can be useful for monitoring slow responding external services for example. Here’s an example of the console output when information level logging is enabled:

info: System.Net.Http.HttpClient.MyClient.LogicalHandler[100]
  Start processing HTTP request GET https://api.github.com/repos/aspnet/docs/branches

info: System.Net.Http.HttpClient.MyClient.ClientHandler[100]
  Sending HTTP request GET https://api.github.com/repos/aspnet/docs/branches

info: System.Net.Http.HttpClient.MyClient.ClientHandler[101]
  Received HTTP response after 682.9818ms - OK

info: System.Net.Http.HttpClient.MyClient.LogicalHandler[101]
  End processing HTTP request after 693.1094ms - OK

If you require a deeper level of detail regarding the requests, this is available at trace level. With trace level logging enabled, details about the request and response headers will also be included in the log messages. Here’s an example from a request with trace logging enabled:

info: System.Net.Http.HttpClient.MyClient.LogicalHandler[100]
 Start processing HTTP request GET https://api.github.com/repos/aspnet/docs/branches

trce: System.Net.Http.HttpClient.MyClient.LogicalHandler[102]
  Request Headers:
  Accept: application/vnd.github.v3+json
  User-Agent: HttpClientFactory-Sample

info: System.Net.Http.HttpClient.MyClient.ClientHandler[100]
  Sending HTTP request GET https://api.github.com/repos/aspnet/docs/branches

trce: System.Net.Http.HttpClient.MyClient.ClientHandler[102]
  Request Headers:
  Accept: application/vnd.github.v3+json
  User-Agent: HttpClientFactory-Sample

info: System.Net.Http.HttpClient.MyClient.ClientHandler[101]
  Received HTTP response after 795.6954ms - OK

trce: System.Net.Http.HttpClient.MyClient.ClientHandler[103]
  Response Headers:
  Server: GitHub.com
  Date: Sun, 08 Jul 2018 09:44:09 GMT
  Status: 200 OK
  X-RateLimit-Limit: 60
  X-RateLimit-Remaining: 58
  X-RateLimit-Reset: 1531046594
  Cache-Control: public, max-age=60, s-maxage=60
  Vary: Accept
  ETag: "f0452653b55e5fef139a58372e3a7bf3"
  X-GitHub-Media-Type: github.v3; format=json
  Access-Control-Expose-Headers: ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
  Access-Control-Allow-Origin: *
  Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
  X-Frame-Options: deny
  X-Content-Type-Options: nosniff
  X-XSS-Protection: 1; mode=block
  Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
  Content-Security-Policy: default-src 'none'
  X-Runtime-rack: 0.029792
  X-GitHub-Request-Id: DCD6:3C9D:688D222:D064A9D:5B41DCE9
  Content-Type: application/json; charset=utf-8
  Content-Length: 2642

info: System.Net.Http.HttpClient.MyClient.LogicalHandler[101]
  End processing HTTP request after 818.4525ms - OK

trce: System.Net.Http.HttpClient.MyClient.LogicalHandler[103]
  Response Headers:
  Server: GitHub.com
  Date: Sun, 08 Jul 2018 09:44:09 GMT
  Status: 200 OK
  X-RateLimit-Limit: 60
  X-RateLimit-Remaining: 58
  X-RateLimit-Reset: 1531046594
  Cache-Control: public, max-age=60, s-maxage=60
  Vary: Accept
  ETag: "f0452653b55e5fef139a58372e3a7bf3"
  X-GitHub-Media-Type: github.v3; format=json
  Access-Control-Expose-Headers: ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
  Access-Control-Allow-Origin: *
  Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
  X-Frame-Options: deny
  X-Content-Type-Options: nosniff
  X-XSS-Protection: 1; mode=block
  Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
  Content-Security-Policy: default-src 'none'
  X-Runtime-rack: 0.029792
  X-GitHub-Request-Id: DCD6:3C9D:688D222:D064A9D:5B41DCE9
  Content-Type: application/json; charset=utf-8
  Content-Length: 2642

As you can see and hopefully would expect, at trace level, the details are quite verbose. This can be useful to gain a more complete understanding of the headers during development. It is not recommended that you enable this in production since it will not only quickly fill logs, but it may expose secure data such as authorisation tokens for example.

Each log message includes an event ID so that you can quickly filter out the events you are interested in. There are two loggers in use by default. There is an outer “LogicalHandler” logger which wraps the entire handler pipeline. This will allow for the timing of the entire pipeline to be included in the logs. Additionally, as this will run first in the pipeline, it will log the request headers as they appear before the request passes through any other handlers. The other handlers in the pipeline may modify those headers. Using the trace level logging it’s possible to capture that information to your logs which can be useful if you need to diagnose failures.

The inner logger has the suffix, “ClientHandler” in the category name. This will be the innermost handler and therefore be the last custom handler to run before the request is sent over the network. As a result, this will be able to record the final request headers as they appear before the request is sent over the wire.

For reference, the event IDs included by these loggers are as follows:

Outer “LogicalHandler”

100 RequestPipelineStart
101 RequestPipelineEnd
102 RequestPipelineRequestHeader
103 RequestPipelineResponseHeader

Inner “ClientHandler”

100 RequestStart
101 RequestEnd
102 RequestHeader
103 ResponseHeader

How does IHttpClientFactory logging work?

The logging in IHttpClientFactory is applied to the pipeline just before HttpMessageHandlerBuilder.Build() method is called to return the final HttpMessageHandler pipeline.

This is achieved with the help of HttpMessageHandlerBuilder filters which are applied by the DefaultHttpClientFactory implementation. An interface named IHttpMessageHandlerBuilderFilter exists which can be implemented in order to define filters. By default, there is one implementation of this interface called LoggingHttpMessageHandlerBuilderFilter which is registered within DI. It’s possible to record more than one implementation against the interface. As long as the implementations are registered in DI, each one will be executed when building the pipeline.

The code in the LoggingHttpMessageHandlerBuilderFilter implementation of the Configure method is responsible for creating the two loggers and passing them to logging handlers, which themselves are implementations of the DelegatingHandler abstract base class. The code for LoggingHttpMessageHandlerBuilderFilter.Configure method is as follows:

When the delegate chain is called, the next(builder) call (line 11) will execute the next delegate (an Action<HttpMessageHandlerBuilder>) to ensure the entire handler pipeline is configured.

Then, the two loggers are created, using the name from the builder. This will be the name given to the named client or the type of the typed client.

The outer handler is added at index 0 to the AdditionalHandlers list so that it surrounds all other handlers and is the first to execute. The inner handler is added to the end of the AdditionalHandlers list, so it will be the last to execute before the internal handlers responsible for making the HTTP request.

Each of these logging handlers is responsible for logging their messages before and after the SendAsync calls to the other handlers. Using this approach a timer can be started before the SendAsync call and used afterwards to record the total HTTP request time as well as the overall handler pipeline execution time. I won’t copy the code for those handlers here as they are quite long. Instead, if you are interested you can view them on GitHub.

The outer “LogicalHandler” source can be viewed at here.

This class creates a logging scope as well as recording the log messages. Optionally, if Trace logging is enabled, it will iterate over the request and response headers, recording those to the logger also. The pattern used in this class is an example of the LoggerMessage approach to provide caching of the logging delegates for better performance. You can read more about this approach in the official documentation at docs.microsoft.com. It’s a little outside the scope of this post to go any deeper here.

The inner “ClientHandler” logger uses a very similar approach to record its log messages and the source for that can be viewed here.

Configuring the logging output

As with all logging which uses the Microsoft.Extensions.Logging library, you can control the log messages that are generated using configuration.

In the appsettings.json file, you can control and filter the logging which is recorded. The default production JSON file looks like this:

In this configuration, only warning messages and higher will be logged and as a result, no logging from the IHttpClientFactory handlers will be included. To enable logging we can add an extra log level configuration setting. If you read the “How does it work” section above you’ll recall that the loggers which are used to log the HTTP request log messages are defined with the category name beginning with “System.Net.Http”.

An option, therefore, is to enable the Information or Trace log levels for the System namespace:

However, this will also enable informational messages from all other components in the System root namespace. Therefore it may be better to configure the logging by limiting the configuration to “System.Net.Http.HttpClient” so that you only add in the messages from the HTTP requests through clients created via the HttpClientFactory:

We can take this filtering a step further and filter down to a specific named or typed client. For a typed client, the name will make the name of the registered type.

Let’s imagine we are only interested in logging the requests via a typed client named MyClient. Also, perhaps we only want the raw timing of the HTTP request itself. In this example we can enable logging just for the ClientHandler of our MyClient:

Customising the log messages

There may be cases where you want to add additional logging around HTTP requests through the clients managed by IHttpClientFactory. An option, in this case, is to introduce an extra handler into the pipeline. In part 3 of this series, I explored adding additional outgoing middleware handlers to your client configuration. Using that handler you can inspect the requests and responses, logging any data as necessary.

If you want to replace the default logging entirely to fully customise the message output, the recommended approach from the team is to replace the default implementation of the IHttpMessageHandlerBuilderFilter interface. In fact, this section of the post was inspired by an issue on the IHttpClientFactory GitHub repository. Let’s take a look at how we replace the logging so that we can record a correlation ID into the console messages. We want to replace the default implementation here since we don’t want additional log messages.

First, we’ll need to create a new implementation of IHttpMessageHandlerBuilderFilter:

I’ve pretty much copied the default implementation for this filter. The main difference is that for simplicity I’m only using one outer logger for this example. We create a logger and then add in a new CustomLoggingScopeHttpMessageHandler to the handler pipeline.

The CustomLoggingScopeHttpMessageHandler class is as follows:

There’s quite a lot to this class, but most of that is the static Log class and its methods. I won’t go into those too deeply here since they following the LoggerMessage advice for more performant logging that you can read in the docs. For the most part, I took the exiting LoggingScopeHttpMessageHandler method and tweaked it for my needs.

The first point to focus on here is that whole operation is wrapped in a logging scope. Before and after the SendAsync method is called on the base, we use the static Log methods to record the log events (lines 20 and 22)

Within the Log class, a few private delegates are defined to format the expected log messages.

Both _beginRequestPipelineScope and _requestPipelineStart accept a string which will be the correlation ID. They use the value to record the correlation ID into the scope properties as well as on the request started message.

A simple helper method has been added which parses a HTTP request for the expected correlation ID header and if present, returns it. The BeginRequestPipelineScope and RequestPipelineStart both use this method to extract the correlation ID.

The final step now that we have our filter implementation is to register it in DI, replacing the existing default filter applied by the HttpClientFactory library.

Inside the Startup class, ConfigureServices method we can call the replace extension on the ServiceCollection to swap out the default implementation with our one:

The replace method will find the first registered service of IHttpMessageHandlerBuilderFilter and replace that registration with this new one, where our CustomLoggingFilter is the implementation.

Now, when we run the application, the console logs include our correlation ID:

info: System.Net.Http.HttpClient.MyClient.LogicalHandler[100]
      Start processing HTTP request GET https://api.github.com/repos/aspnet/docs/branches [Correlation: 447c8d6b-e280-4538-bd31-56d508266b5b]

info: System.Net.Http.HttpClient.MyClient.LogicalHandler[101]
      End processing HTTP request - OK

As a side note, this filter approach is a great way to add common cross-cutting concerns for your whole application. It’s possible to register additional filters, each of which could add their own common handlers onto all clients created via HttpClientFactory.

Summary

In this post, we looked at the type of information available to us from the built-in logging, included as part of the HttpClientFactory library. We looked at how we can use log configuration to control which log messages we see and also looked at how logging has been implemented within the library. Finally, we explored using the IHttpMessageHandlerBuilderFilter interface to replace the default logging filter. I hope this has been helpful. I’ll be keeping an eye on the progress for the 2.2 release where we may see more logging, including some Polly integration making its way into the product.

Other Posts in this Series

Part 1 – An introduction to HttpClientFactory
Part 2 – Defining Named and Typed Clients
Part 3 – Outgoing request middleware with handlers
Part 4 – Integrating with Polly for transient fault handling
Part 5 – This post

IHttpClientFactory Patterns: Using Typed Clients from Singleton Services Exploring an approach to reuse transient typed clients within singleton services

I’ve been following IHttpClientFactory for some time and have created a number of blog posts on the various features based on sample applications. Since then; with the release of ASP.NET Core 2.1 now available, I’ve begun using IHttpClientFactory in real production applications. In this post, I want to start covering some patterns I’ve begun to apply as I develop my applications.

This time I want to look at an issue regarding the scope of typed clients. This is based on something I’ve now directly encountered and that was raised as a comment on an earlier post by a reader named Yair.

When defining typed clients in your ConfigureServices method, the typed service is registered with transient scope. This means that a new instance is created by the DI container every time one is needed. The reason this occurs is that a HttpClient instance is injected into the typed client instance. That HttpClient instance is intended to be short lived so that the HttpClientFactory can ensure that the underlying handlers (and connections) are released and recycled.

This works in cases where you plan to consume the typed service from another transient service. A common place to use these in ASP.NET Core will be places such as Controllers for example. That works as expected since the controller is created by the framework for each request.

However, what if you want to use the typed client within a singleton service? That presents a problem. If this was allowed it would be created and injected once, and then held onto by the singleton service. This is not the behaviour we want.

Originally, when asked by Yair about how to use HttpClientFactory in singleton services I suggested instead using the named client approach and then injecting the IHttpClientFactory directly to your singleton service. From there, you can call CreateClient on the factory within methods on that singleton service, so that for each invocation, a new HttpClient is created for only a short lifetime.

The problem is that if we don’t get the typed client behaviour where we can encapsulate the work necessary to interact with a third party API as a service. Instead, we would have to make a service that depends on the IHttpClientFactory as I suggested above and then pass that into the necessary places in our code.

Having now tackled this in a real project where I am essentially building an SDK around an internal API that I’ve developed I have reassessed the options. What I ended up doing was using the typed client approach, but also providing my own factory which can return instances of that typed client. This is actually very simple to do by leveraging the DI service provider directly.

Here is an example of a typed client and its interface:

I won’t dive deeply into how this service works. It’s a typed client and you can read my earlier post about named and typed clients for more information.

In short; this service expects to have a HttpClient instance injected when it is created by the DI container. It then wraps the logic needed to call various endpoints of a remote API. Within this service, we can include the code needed to validate the response and deserialise the returned content from the request.

We can register the typed client in our ConfigureServices method as follows:

At this point, we have a typed client which can be consumed from other transient services and controllers. To make this accessible to singleton services in our application we can add a basic factory.

Here we have created a basic interface for an IConfigurationServiceFactory which defines a single GetConfigurationService method.

The implementation takes an IServiceProvider in its constructor, which will be injected by DI. With access to the service provider, we can use it to return an instance of IConfigurationService from the GetConfigurationService method. As this is a transient typed client, a new instance will be returned each time this method is called.

In ConfigureServices, we can register the factory with DI as a singleton:

As this is a singleton, we can consume this from any class where we need access to an instance of the typed client, even if that class is registered with singleton scope in DI.

The important thing here is that we don’t create an instance of the IConfigurationService and hold onto it for the lifetime of the singleton service. We can hold the IConfigurationServiceFactory and then we must use that whenever a method needs to get access to the IConfigurationService.

I’m sure there may be other ways to achieve similar results but I’m fairly happy with this approach for now.

Related Posts

Part 1 – An introduction to HttpClientFactory
Part 2 – IHttpClientFactory – Defining Named and Typed Clients
Part 3 – IHttpClientFactory – Outgoing request middleware with handlers
Part 4 – IHttpClientFactory – Integrating with Polly for transient fault handling

How to record metrics to DataDog from ASP.NET Core Recording metrics and events to DataDog in development and production in AWS ECS.

I’m currently architecting and building a new microservices based system at work. A priority of mine has been to learn from the experience of building our first microservices project by putting a greater focus on logging and monitoring. We freely acknowledge that we didn’t get this as good as we would have liked in our first implementation. Logging is hugely important in a microservices design. Tracking down failures and bugs can be very difficult without good logging and even more importantly, a way to search and view those logs.

But logging isn’t what I want to cover in this post. Alongside logging, something we felt we were missing in our previous project was detailed performance and usage metrics. We can gain some insight into the state of the system using built-in AWS metrics for our ECS services and also by monitoring, again via CloudWatch things such as SQS queue metrics. This is all reasonably useful but at times it would be great to know a bit more about the services themselves.

With this new project, I’ve started instrumenting it with DataDog so that the application can record useful metrics that allow us to track it’s overall health, as well as understand how things are performing and what’s actually being used. The goal is to build up a much better profile of the application so that we can more easily see whether changes that we have deployed have improved the performance or whether they have degraded it. Having proper baselines of the application performance is key to being able to make such determinations.

The choice of DataDog was made, simply because our systems team use it already for monitoring the overall AWS environment. After a few quick enquiries, I found that recording metrics from the application code would be possible thanks to the availability of a C# library from DataDog called “dogstatsd-csharp-client”. My only concern is that the library seems to be a little stalled, with no commits since Nov 30th 2017. There is also a third party library called DatadogSharp which I may investigate in the future. Their readme suggests it is more performant than DataDog’s own library. Putting that aside for now; I decided I’d go ahead and prototype based on the DataDog library and revisit these concerns if they proved founded, once I was underway. I have built a small wrapper library over the static DogStatsd methods so I have a layer of abstraction if I decided to change the underlying client. It also allows me to centralise the use of some common tags which I wanted to include on all of my metrics.

In this post, I want to focus on the slightly higher level concepts by discussing how I got this working during development and more recently, deploying to a prototype in production running in AWS ECS.

Development

The DogStatsd client sends messages over UDP to an agent server which will collect these and eventually send them up to the DataDog service. You can read more about that flow on the DataDog DogStatsD page. Therefore, to test my code during development, I needed to have an agent server running somewhere.

In my case, I opted for a Docker image which I could run locally. After a bit of Googling, I ended up at the DogStatsD6 Docker Image GitHub repository. This image is hosted on the Docker Hub in the DataDog DogStatsD repository.

To begin working with this locally, I created a simple docker-compose file. I’ve not included some other elements I was running here for logging via the ELK stack as those are not necessary for this example. My docker-compose file looked like this:

The above example specifies a dogstatsd service which will be started using the DogStatsD6 image. By default, this DogStatsD agent server will be listening on UDP port 8125 for data. I expose that port to port 8125 on my host (development) machine. For the DogStatsD server to be able to send its data to DataDog we need to provide an API key. This can be done by passing the key into the DD_API_KEY environment variable.

At this point, I could run a docker-compose up -d command to start an instance of the DogStatsD container.

Sending Metrics and Events from an ASP.NET Core based API

As I mentioned earlier, I’ve created a wrapper library for sending DataDog metrics and events via the methods from the dogstatsd-csharp-client library. I won’t be covering that here as I want to keep this post focused on the core principles. Instead, let’s look at how we can begin with recording metrics and events to DataDog.

The first step, before we can send data to DataDog is to add the Nuget package for the dogstatsd-csharp-client library.

You can do this by searching for it via the NuGet package manager, or as I did, by editing the csproj file directly and adding the package reference.

<PackageReference Include="DogStatsD-CSharp-Client" Version="3.1.0" />

Once the library is added we must configure the client. All of the functionality for this client library is implemented as static methods. The first method we will call is the DogStatsd.Configure method. This takes a StatsdConfig object as a parameter. We’ll add a small static method in our Startup.cs class as follows:

We build up the configuration object using a server name string which we will pass into this helper method. This will be the hostname or IP of the server running the DogStatsd server. We also set the port it will be listening on. Finally, we can add a prefix which will be added to the front of any metric and event names. This is useful to make it easier to search for them in the DataDog application.

We then call the Configure method, passing in the config object.

Next, we need to call this method. We can do that from our Configure method in the Startup class. This is called once at application startup so is an appropriate place to do this. First, we’ll load the DogStatsd server name from ASP.NET Core configuration as this allows us to easily set it per environment. We’ll then call our ConfigureDataDog method, passing that server hostname through:

We’ll need to include a section for this configuration in our appsettings.json file:

During development, we can set this to the localhost IP address (127.0.01). Since we are running are DogStatsd server within a container and exposing its port through to the host we can access it this way. In production, we can then set an appropriate production hostname for the DogStatsd server.

At this point, everything is ready to go. We can now use the static DogStatsd methods to send metrics and events from our code.

The client library readme includes examples of the methods we can call but as a basic example, here’s some code we can call from our application to record a metric whenever a profiles endpoint is called.

DogStatsd.Increment("all_profiles_endpoint_call");

At this stage, we have added basic metrics to our application and have a DogStatsd server running locally in a container to collect and forward those metrics to DataDog.

Production

NOTE: Since trialling this approach, we have learned that because each sidecar is considered a new host, this may have cost implications with the DataDog pricing model. We are moving away from this approach to daemon mode containers (1 per ECS host). I will update accordingly once we are happier with that approach. For now, this stills works, but do consider how it affects your billing.

The final stage is to get this working in production. I felt there might be a few options here but the one I wanted to try was to run the DogStatsd server as a sidecar container. We use AWS Elastic Container Service (ECS) to run our production services as Docker containers.

To achieve this we will add a second container to our task definition for our ECS service. ECS services are the logical structure that represents a unit of scale and deployment for containers in ECS. The service can run multiple containers which generally is not something you need or want to do since it limits your scaling options. However, for this requirement, running a second supporting container, it is a reasonable use case. This container is directly used only by our main microservice and should scale with that service.

The final ECS task definition looks like this:

The new section in my case is the dogstatsd container definition near the bottom (lines 39-51). The task definition takes one or more container definitions which define the containers that will be started as part of the tasks for this service.

The dogstatsd one is quite simple. We tell it to use the same image as we used in development, “datadog/dogstatsd”, which will be pulled from the Docker Hub. We can pass in the API Key via an environment variable. We can mark this container as non-essential since we don’t want its failure to cascade to causing our service to restart unexpectedly.

The next change is to add the links to the main API server container definition (lines 35-37). This ensures that it will have a network link with the dogstatsd container. Finally, we can add an environment variable for the API container which sets the hostname for the DogStatsD server. This will override the 127.0.0.1 setting from our appsettings.json file. Since we have linked the containers we can reference the DogStatsd server by its container name.

With these changes made it was then a case of registering this new task definition and updating our ECS service. When the service starts tasks, it will start an instance of the DogStatsd container which our application containers can use to record their metrics and events.

Summary

It’s still early days, but this is working quite nicely in the production environment. I will be reviewing it over time as I learn more about the various elements at play. This is likely not the end of the story or our journey with DataDog. Hopefully, this is enough information to get you started if you are following a similar path.