Implementing ASP.NET Core Automatic Span (Activity) Linking for Internal Redirects with Middleware on .NET 9

Today, I’ll continue a current theme for my content based on my experiences implementing OpenTelemetry instrumentation in practice for .NET applications. In this post, I want to focus on a minor enhancement I recently added to a project that enables span links between request traces on ASP.NET Core during internal redirects.

NOTE: This code relies on a new API added in .NET 9, which allows us to add links to an existing Activity after creation. Therefore, it will not work on applications built on .NET 8 and earlier.

It’s not uncommon for us to want to redirect the user to an alternative endpoint within our application during request handling. One example includes the OAuth flow when authenticating a user via a third-party identity provider. We may receive several HTTP requests in such cases as the user flows through authentication. The first redirect may be from an endpoint requiring an authorised user to send the browser to the login page. Another may include handling the OAuth callback before redirecting the user to their initially requested destination.

In these situations, I wanted to trace the causality of requests such that when viewing a trace for one request, I could navigate to the original request that issued the redirect. When a redirect is returned that the browser follows, the headers used for trace propagation are no longer helpful. We could tackle passing the trace context through the redirect in a few ways. We could include traceparent information on the query string or in a cookie, for example. With ASP.NET Core, we can also leverage the TempData mechanism (which by default uses a cookie-based provider) to preserve data across requests, which is the approach I’ll use for this post.

NOTE: The code shown in this post is for illustrative purposes. While I expect it to work reasonably well, it has not been extensively battle-tested. Feel free to copy and paste it, but take care to validate it before deploying it to production!

Let’s begin with the initial outline of the middleware. We’ll build this up in stages so I can explain each code block as we add it.

public class RedirectActivityLinkingMiddleware(
    RequestDelegate next, 
    ITempDataDictionaryFactory? tempDataDictionaryFactory)
{
    private readonly RequestDelegate _next = next;
    private readonly ITempDataDictionaryFactory? _tempDataDictionaryFactory = tempDataDictionaryFactory;

    private const string RedirectParentActivityId = nameof(RedirectParentActivityId);
    private const string RedirectParentActivityIdSet = nameof(RedirectParentActivityIdSet);

    public async Task Invoke(HttpContext context)
    {
        var tempData = _tempDataDictionaryFactory?.GetTempData(context);

        if (tempData is null)
        {
            await _next(context);
            return;
        }

        // more code to follow…

        await _next(context);
    }
}

This implementation relies on the availability of an ITempDataDictionaryFactory, which should be available from the service provider for applications created from the ASP.NET Core templates. If that service is unavailable, we no-op and invoke the next RequestDelegate in the pipeline.

In the commented section, we’ll add some logic to store the current activity ID to retrieve it on the subsequent request.

context.Response.OnStarting(static state =>
{
    var context = (HttpContext)state;

    var activity = context.Features.Get<IHttpActivityFeature>()?.Activity;

    if (activity is null)
        return Task.CompletedTask;

    var statusCode = context.Response.StatusCode;

    if (statusCode == 301 || statusCode == 302 || statusCode == 303 || statusCode == 307 || statusCode == 308)
    {
        var location = context.Response.Headers.Location;

        if (!location.ToString().AsSpan().StartsWith('/'))
            return Task.CompletedTask;

        var factory = context.RequestServices.GetService(typeof(ITempDataDictionaryFactory)) as ITempDataDictionaryFactory;
        var tempData = factory?.GetTempData(context);

        if (tempData is not null && activity is { Recorded: true })
        {
            tempData[RedirectParentActivityId] = activity.Id;
            tempData[RedirectParentActivityIdSet] = DateTimeOffset.UtcNow.ToString();

            tempData.Save();

            context.Response.Headers.Location = location.Contains("?")
                ? (location += "&redirect=true")
                : (location += "?redirect=true");
        }
    }

    return Task.CompletedTask;
}, context);

This next important block of code registers a delegate for Response.OnStarting. Our logic needs to know the status code and headers for the response to function, but it also needs to add its own header. We can achieve this by hooking into OnStarting, because our header needs to be added before the response body is sent. This code is responsible for accessing the current trace information and for redirects, adding that to the temp data to access the ID for span linking on the subsequent request. This code uses an overload that accepts an object, providing the state used within the lambda. This avoids creating a closure and is, therefore, slightly more performant. As such, we can apply the static keyword to ensure we don’t capture any variables from outside the scope of the lambda code. In this case, the state we pass in is the HttpContext from the current request.

For this code to work, we need to access the current Activity (think span in OpenTelemetry terminology) for the request that will issue the redirect. We can retrieve this via the IHttpActivityFeature added to the HttpContext for the request. The activity may be null if there is no observer for the ASP.NET Core instrumentation code, so we must handle that scenario.

We also need to access the status code set on the response. We’ll check if this code matches any potential HTTP status codes for a redirect. We don’t want to set the TempData data for regular non-redirect responses.

A second check we make is to avoid setting TempData for redirects outside our application. We only want to attempt to set this data when we know the user will be immediately redirected to a different endpoint on our site. This is partly to avoid undue overhead when the data is not useful and secondarily to prevent incorrect links from being attributed to subsequent requests. The code checks to see if the URL starts with a forward slash to achieve this.

We’ll need access to the ITempDataDictionary, which can be accessed via the ITempDataDictionaryFactory. We resolve that from the RequestServices in this block. We could also have considered passing this as state, perhaps using a ValueTuple, although that would have incurred boxing, which we avoid in this code.

We call GetTempData on the ITempDataDictionaryFactory to get an instance of ITempDataDictionary. As long as that is not null, we add two items to the dictionary. The first is the ID of the current request Activity, equivalent to the value used for a standard traceparent header in distributed trace propagation. The second thing we add is the millisecond epoch timestamp when adding the temp data. This will be useful later to avoid us incorrectly attributing links to future requests where something strange happens, and there’s a long delay. After adding the two items, we save the temp data.

The final piece of code appends a query string argument to the URL. This is not absolutely necessary but is a safety mechanism to ensure that we only load the temp data when handling a request which includes it. This mitigates an edge case where a user may browse the application from two tabs, and we may accidentally associate an incorrect link. In reality, this is so unlikely due to the time limit. This is, therefore, something I won’t bother with in my final application. I dislike ” polluting” the URL to mitigate this scenario. In the worst case, we very rarely get an unplanned link between page views. I’ve left it here as an example should this risk be something you want/need to avoid.

We’re left with one last chunk of code that will handle the span linking for requests.

var activity = Activity.Current;

if (activity is { IsAllDataRequested: true } 
    && tempData.TryGetValue(RedirectParentActivityId, out var tempDataObject)
    && tempData.TryGetValue(RedirectParentActivityIdSet, out var tempDataSetObject)
    && tempDataObject is string parentId
    && tempDataSetObject is string tempDataSet
    && ActivityContext.TryParse(parentId, null, isRemote:false, out var ctx)
    && DateTimeOffset.TryParse(tempDataSet, out var dateSet))
{
    var millisecondsDifference = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds() - dateSet.ToUnixTimeMilliseconds();

    if (millisecondsDifference < 2000 
        && context.Request.Query.TryGetValue("redirect", out var value) && value.Equals("true"))
    {
        activity!.AddLink(new ActivityLink(ctx));
    }
}

tempData.Remove(RedirectParentActivityId);
tempData.Remove(RedirectParentActivityIdSet);

await _next(context);

This code first accesses the current Activity and checks its IsAllDataRequested property returns true, meaning the Activity is sampled and should be enriched. It then accesses the ITempDataDictionary to try and extract the activity ID and the timestamp when it was set. The remaining conditions ensure we can cast and parse the object for the value into the expected types.

If so, we first check that this request was made within 2000 milliseconds of the redirect. This is another value you may want to tweak. I felt 2 seconds was sufficient for the redirect to hit the browser and the new page to be requested. Hopefully, it’s short enough to avoid misappropriating the link. We don’t want the data to be valid for too long in case the user never gets redirected for some reason but makes a new request of their own to another URL. Our additional safety net comes into play, too, as we check that the URL for this request includes our “redirect” query string entry. This at least ensures we only try to add links to pages we know to have resulted from an internal redirect.

After validating all the conditions, we call the new .NET 9 AddLink API on the Activity instance. We must pass an ActivityLink to this, which we can create from the parsed ActivityContext.

In all scenarios, we remove the two temp data elements. This shouldn’t strictly be necessary as these get removed after we read them, but I added them to ensure they are gone as soon as possible. Feel free to skip those lines of code.

And that’s it; we now have a single middleware that ensures all requests triggered by an internal redirect can be linked to the originating request. This can be useful to track and is another way to enrich the value of tracing data that applications emit.

With the middleware defined, we can add it to the request pipeline. We’ll want to do this early on before most other middleware runs.

var app = builder.Build();

if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler("/Home/Error");
    app.UseHsts();
}

app.UseHttpsRedirection();

app.UseMiddleware<RedirectActivityLinkingMiddleware>();

// Any other application-specific middleware

app.MapDefaultControllerRoute();
app.Run();

Considerations

I’ve tried to be over-cautious in the above code to prevent invalid data. There are still edge cases where this could fall down, but those seem unlikely for a regular browsing session. As a result of the choices above, span links might be missed for high-latency scenarios where the redirect isn’t received within the timeframe you expect. The timeout could be increased if those occur frequently. It would even be possible to consider adding metrics to tally the setting and retrieval of these temp data values that could be used to tally whether any appear to be lost.

Conclusion

In this post, I presented one possible way to achieve span (Activity) linking between a request resulting in an internal redirect and the request for the redirect location. It has a few minor trade-offs, and in very rare edge cases, it might result in an incorrect link, but I think it’s sufficient for most scenarios. It feels like a reasonable way to enrich spans to show the causal relationship between multiple requests.