Health Checks with gRPC and ASP.NET Core 3.0 Header

Health Checks with gRPC and ASP.NET Core 3.0

In this post, we will explore a couple of options to apply health checks to your gRPC server running on ASP.NET Core 3.0. Health checks are a common requirement. In particular, in containerised environments, they are required so that the container orchestrator and load balancers know which services are actually functioning.

This post is part of my gRPC and ASP.NET Core series.

Option 1: Using the ASP.NET Core Health Check Middleware

The first quite simple option is to leverage the exiting health check middleware which is documented within ASP.NET Core docs. This middleware supports various configurations to suit many requirements. It adds at least one health check endpoint to your application, reachable via a regular HTTP request.

For a basic health check, we can add the feature to an ASP.NET Core application by first registering the health check services within the ConfigureServices method.

services.AddHealthChecks();

We then map a health check endpoint under the /health path using the IEndpointRouteBuilder.

On line 4 of the above code, we use the MapHealthChecks method to register a simple health check endpoint.

When our application is running, we can make a get request to the configured endpoint. On my local machine, I can request “https://localhost:5005/health” which returns a 200-OK response with “Heathy” as the response body.

I won’t go into more advanced configurations for the health check feature in this post.

Option 2: Using the Health Checks Library

The gRPC framework specifies a gRPC health checking protocol. It uses the same approach of defining a service definition as for regular gRPC services. Health checking is just another set of RPC methods.

A common service definition is available and defined using a proto file called health.proto, in the proto package grpc.heath.v1.

The service definition includes two RPC methods currently:

Check‘ is a unary service method which returns a HealthCheckResponse when a HealthCheckRequest is received.

The HealthCheckRequest is defined as follows:

The request includes a single string field called ‘service‘. A gRPC server may be handling multiple gRPC service definitions. gRPC health checks support the concept of requesting a health check status for each specific service. The string is used to identify the service for which the health check status should be returned.

The ‘Check‘ method returns the appropriate HealthCheckResponse if the provided service name is known. In cases where it is not known, the response from the gRPC server will be NOT_FOUND.

The response message has the following definition:

The ‘ServingStatus‘ enum is used to define some possible statuses that may be returned. Serving indicates that the server is running and ready to handle calls for the service. NOT_SERVING identifies that the opposite i strue and that the server is not currently healthy.

The ‘Watch‘ method defines a way for the server to stream status changes for a given service. When called from a client, an immediate response will be returned indicating the current status, which may be NOT_SERVING or SERVICE_UNKNOWN, for example. The call will remain open so that if the status changes on the server, an updated response will be sent to the client.

A server can provide an implementation for this service definition to support health check RPC calls from clients and infrastructure such as load balancers. We could, of course, implement this ourselves but for basic requirements the Grpc.HealthCheck NuGet package can be included in your server project. This package contains an implementation of the ‘Check‘ method that we can utilise.

To get started, add a package reference to your server project using your preferred approach. Once complete, you should have a PackageReference included in your project file (csproj).

<PackageReference Include="Grpc.HealthCheck" Version="2.24.0" />

This package includes a partial implementation of the health.proto service definition in the class HealthServiceImpl. This class includes an implementation for only the Check method currently. The class defines three other methods.

  • public void SetStatus(string service, HealthCheckResponse.Types.ServingStatus status);
  • public void ClearStatus(string service);
  • public void ClearAll();

These methods are used to register and update the state of the various gRPC services for the server. Internally, a Dictionary is used to maintain the values for each registered service. Our job is to ensure we set the appropriate status(es) and maintain them.

It’s not immediately obvious how best to achieve this. In the end, I settled on using a BackgroundService, a feature of the .NET Core hosting model. We’ll use a basic approach in this post to demonstrate a way to get started.

This class extends the BackgroundService provided by the .NET Core hosting libraries. It supports dependency injection, and we expect two services to be injected into the constructor. The first is the HealthServiceImpl which comes from the Grpc.HealthCheck NuGet package we included earlier. The second is the HealthCheckService from the Microsoft.Extensions.Diagnostics.HealthChecks package.

In our override for the ExecuteAsync method, we include some logic which sets the statuses for our services. This occurs in a while loop which will exit only if the application is shutting down, signalled by the CancellationToken.

We use the CheckHealthAsync method on the HealthCheckService to request the general health status for the application. In more complex, real-world applications, there may be many other things which have to be checked to establish a healthy state; such as the ability to connect to a database or reach a downstream dependency. It is possible to include those checks within the broader Microsoft.Extensions.Diagnostics health checks or as additional, gRPC specific code in this method.

Based on the returned HealthStatus from the CheckHealthAsync method, we then set the appropriate ServingStatus on our gRPC service.

The gRPC Health Checking Protocol specifies that the server should also return the overall server status as a response to a service name that is an empty string. We, therefore, ensure we include a status for both the “Weather” service and, for an empty string, the overall server.

Our ExecuteAsync code then delays (asynchronously) for 15 seconds, before the loop continues and the statuses may update. This is basic logic which ensures that the statuses are refreshed periodically.

Registering and Mapping the gRPC Health Check Implementation

Finally, we must register and map an endpoint to the health service. To support our approach here, we have to do some extra work. Since we need the statuses, set through the HealthServiceImpl class to be globally available and consistent within the application, we need to manually register this class as a singleton service with the dependency injection container. Additionally, we must register our IHostedService so that it is started when the application hosting begins.

We achieve this in the ConfigureServices method of the Startup class.

The first line registers the HealthServiceImpl as a singleton. This ensures that the same instance, with the same dictionary of statuses, is used throughout the application. Internally, the gRPC libraries, which activate the appropriate service to handle calls for an endpoint, will use an instance from the dependency injection container if one is available.

The second line uses the AddHostedService extension method to register our StatusService.

 We can now update our endpoint registration:

On line 5, we include a call to MapGrpcService for the HealthServiceImpl. This ensures that it will be serving for calls to the health check RPC methods.

Calling Health Checks From a Client

Now that our server is configured for gRPC health checks, we can test it by creating a basic console client. We’ll use a new .NET Core 3.0 console application for simplicity.

After creating the console application, we can reference the same Grpc.HealthCheck NuGet library that we added to our server. This includes a client implementation also.

<PackageReference Include="Grpc.HealthCheck" Version="2.24.0" />

We can now use its client to call the health check method.

This simple client implementation establishes a HealthClient using a gRPC channel. The HealthClient has been generated based upon the health.proto service definition. We can, therefore, call the CheckAsync method, providing a HealthCheckRequest, as seen on line 7 in the preceding code. We may supply a specific service name or pass an empty string to request the overall server status.

Summary

In this post, we’ve explored a couple of approaches to including health checks for an ASP.NET Core 3.0 application which acts as a gRPC server.

The first option leverages the existing Health Checks middleware for ASP.NET Core which responds to a GET request made to an HTTP endpoint.

The second option is slightly more complicated but supports the gRPC Health Checking Protocol. Most of this is available as part of the Grpc.HealthCheck NuGet package, although a little work is needed on our end to set the statuses and register things into dependency injection.

Both approaches may be valid, and they can also be combined within a single application if you desire.

You can find a working example of a gRPC with health checks included in my gRPC Demos GitHub Repository.

For more gRPC content, you can find all of my posts that are part of my gRPC and ASP.NET Core series.

Steve Gordon

Steve Gordon is a Microsoft MVP, Pluralsight author, senior developer and community lead based in Brighton. He works for Madgex developing and supporting their data products built using .NET Core technologies. Steve is passionate about community and all things .NET related, having worked with ASP.NET for over 15 years. Steve is currently developing cloud-native services, using .NET Core, ASP.NET Core and Docker. He enjoys sharing his knowledge through his blog, in videos and by presenting at user groups and conferences. Steve is excited to be a part of the .NET community and founded .NET South East, a .NET Meetup group based in Brighton. He enjoys contributing to and maintaining OSS projects, most actively helping save lives with open source software and the Humanitarian Toolbox (www.htbox.org). You can find Steve online at his blog www.stevejgordon.co.uk and on Twitter as @stevejgordon