In this multi-part blog series, we’ll build a reasonably full-featured code base for automatically encrypting/decrypting specific properties on types that are being serialised and deserialised using System.Text.Json. In future posts, we’ll eventually build up to a solution that utilises Azure Key Vault to secure the encryption key protecting the data. Today, we’ll begin with some more fundamental concepts and fake the actual encryption and decryption.
There are two main scenarios I have for encrypting properties on a JSON object. The first and probably most common requirement would be securing sensitive data when writing to or reading from a document database. A canonical example is storing credit card information. Some commerce platforms allow users to store their card details for future purchases. We would not want such sensitive details to be exposed, even if someone gains access to the database; therefore, it would be prudent to encrypt them so that only the application can access them, and we can audit when and how that decryption occurs. There are many kinds of sensitive data that may benefit from being encrypted in datastores as part of a defence-in-depth strategy.
A second scenario, subtly different from the first, is to encrypt properties on long-term, immutably stored data, such as events (when event sourcing, in particular), or perhaps even audit and/or log records that contain PII. This scenario is similar to the first, but the intent here would be to enable crypto shredding should the user ever assert their right to be forgotten. By encrypting the PII data with a unique encryption key per user, we can simply delete the encryption key when “forgetting” the user to make the data irrecoverable, while not having to mutate the originally stored data. Such scenarios are an important consideration for compliance with regulations such as GDPR.
In my case, I want to securely store the OAuth access and refresh tokens for a user session after they authenticate via GitHub. For my application, I’m using Elasticsearch as the main datastore, so ultimately the data is stored as JSON. After a user authenticates with GitHub, we exchange a code with GitHub to retrieve an access and refresh token. To avoid doing this per request, the application stores the authentication session in Elasticsearch. These tokens are sensitive because they may allow anyone with access to them to act on behalf of the user, so we must ensure they cannot be easily accessed or leaked.
There are a few ways we could tackle this requirement. The design we will use involves defining an attribute to mark properties on our type that we want to encrypt during serialisation, and later decrypt when deserialising from JSON. We’ll then implement an IJsonTypeInfoResolver that will be responsible for triggering encryption and decryption of attributed properties.
In this post, I’ll intentionally gloss over some things that we’ll rely on later. I’ll introduce those in more detail when their purpose becomes relevant.
I’m going to use a minimal API project when implementing this feature for the purpose of this blog post. This will make it easy to execute the code and validate the behaviour.
I’m not going to explain that code, but it basically includes a single endpoint that will accept a simple payload, creating an instance of our type to (de)serialise and then test that we can round-trip the encrypted and decrypted values:
using JsonEncryptionBlogExample;
using Microsoft.AspNetCore.Mvc;
using System.Text.Json;
using System.Text.Json.Serialization.Metadata;
var builder = WebApplication.CreateBuilder(args);
// TODO - Wire up services
var app = builder.Build();
app.MapPost("/", static ([FromServices] JsonSerializerOptions jsonOptions,
[FromServices] ILogger<Program> logger, [FromBody] PlainTextDto data) =>
{
var encryptedData = new EncryptedData { AccessToken = data.AccessToken };
var encryptedDataJson = JsonSerializer.Serialize(encryptedData, jsonOptions);
if (logger.IsEnabled(LogLevel.Information))
logger.LogInformation("Encrypted JSON: {Json}", encryptedDataJson);
var decryptedData = JsonSerializer.Deserialize<EncryptedData>(encryptedDataJson, jsonOptions);
if (decryptedData is null || decryptedData.AccessToken != data.AccessToken)
{
throw new InvalidOperationException("Decryption failed");
}
if (logger.IsEnabled(LogLevel.Information))
logger.LogInformation("Decrypted access token: {AccessToken}", decryptedData.AccessToken);
// For demo: return both encrypted and decrypted values
return Results.Json(new
{
EncryptedJson = encryptedDataJson,
DecryptedAccessToken = decryptedData.AccessToken
});
});
app.Run();
internal record PlainTextDto(string AccessToken);
internal sealed class EncryptedData
{
[EncryptedData("AccessTokenKey")]
public required string AccessToken { get; init; }
}
This attribute expects a key name, which, for now, we won’t use. In the future, this will provide a mechanism to choose which encryption key should be used when encrypting the property. There are scenarios where we may need to use different encryption keys for different types of data, and also so that we have the possibility of applying crypto shredding by using a key per user. Otherwise, this is a very simple attribute that can target properties.
Next up, we’ll implement the logic for performing the “encryption”. I use quotes here because, for now, we’re not going to encrypt anything today, just put the pieces in place so that we can in a future post.
There are several choices when it comes to customising serialisation of types and their properties. In the past, I’d have reached for a custom JsonConverter, and that might have worked. However, a more flexible choice available since .NET 7 is to customise using a JSON contract. The contract exists per type being serialised and essentially stores information required for serialisation to function. The contracts are cached to improve the performance of serialisation.
There are two main ways to customise the contract creation: by providing an implementation of JsonTypeInfoResolver, which can be a little complicated, or by providing a modifier for the default resolver. We’re going to use the latter as it’s a little easier to implement. The basic idea is that the default resolver will apply any modifiers we register in order, and those modifiers can update the contract for specific types and their properties.
One important note for the design I’m going to demonstrate is that I know in advance that at some point, I will need to inject dependencies into the modifier to handle the actual encryption/decryption. The code I show now takes that into consideration, but, for now, doesn’t require injected dependencies. We will, however, resolve the modifier instance from the DI container so that we can later inject things.
We’ll define a type named EncryptedJsonTypeInfoResolverModifier and include a method EncryptStringsModifier that takes a JsonTypeInfo, the current contract. I’ll include two local functions inside EncryptStringsModifier that will handle the “encryption” and “decryption”. For the purpose of the example at this stage, these will just append and remove a suffix to identify when a value is “encrypted”.
using System.Text.Json.Serialization.Metadata;
namespace JsonEncryptionBlogExample;
public sealed class EncryptedJsonTypeInfoResolverModifier
{
private const string EncryptionSuffix = "_encrypted";
public void EncryptStringsModifier(JsonTypeInfo jsonTypeInfo)
{
ArgumentNullException.ThrowIfNull(jsonTypeInfo);
// TODO
// Demo encryption: append suffix
static string Encrypt(string plainText) => plainText + EncryptionSuffix;
// Demo decryption: remove suffix if present
static string? Decrypt(string? cipherText)
{
if (string.IsNullOrEmpty(cipherText) || !cipherText.EndsWith(EncryptionSuffix))
return cipherText;
return cipherText[..^EncryptionSuffix.Length];
}
}
}
JsonTypeInfo to apply the encryption/decryption behaviour to specific properties.EncryptedDataAttribute applied to them. We’ll add the following code to the beginning of the EncryptStringsModifier method:
if (jsonTypeInfo.Kind != JsonTypeInfoKind.Object)
return;
foreach (JsonPropertyInfo jsonPropertyInfo in jsonTypeInfo.Properties)
{
// Only support string properties for now
if (jsonPropertyInfo.PropertyType != typeof(string))
continue;
var attribute = jsonPropertyInfo.AttributeProvider?
.GetCustomAttributes(typeof(EncryptedDataAttribute), inherit: false)
.OfType<EncryptedDataAttribute>()
.SingleOrDefault();
if (string.IsNullOrEmpty(attribute?.KeyName))
continue;
var propertyGet = jsonPropertyInfo.Get;
var propertySet = jsonPropertyInfo.Set;
if (propertyGet is null || propertySet is null)
continue;
// Cache metadata outside the delegate
var shouldSerialize = jsonPropertyInfo.ShouldSerialize;
var isGetNullable = jsonPropertyInfo.IsGetNullable;
// TODO: Encrypt on serialization
// TODO: Decrypt on deserialization
}
JsonTypeInfo represents the contract we have for the current type so far, generated by the internals of System.Text.Json. We can access the Kind property to learn more about the kind of metadata the current contract specifies. The possible values here are “None”, “Object”, “Enumerable”, and “Dictionary”. If the type being serialised is a collection or dictionary, the enumerable or dictionary kinds are used, respectively. None represents a simple value or a type using a custom converter. The kind we’re interested in for now is “Object”.JsonPropertyInfo type. We can first check if the PropertyType identifies this property is typed as a string, which, for now, is the only type we’re going to handle encryption for. For any other type of property, we skip past it.AttributeProvider, we call GetCustomAttributes, passing the type of our custom attribute, and for the second argument, we pass false, to indicate we don’t want to inherit the attribute. In a future iteration, it may be useful to inherit the attribute from the object if all properties should be encrypted. We expect at most one attribute. If the attribute is present on the property, we validate that it has a non-null, non-empty key name.JsonPropertyInfo also stores a delegate representing the getter and setter for the property. These may be null if the property does not include an available getter or setter. We need both to be present to support our implementation, as we’ll need to get the unencrypted value during serialisation and be able to set the property with the decrypted value during deserialisation.JsonPropertyInfo, we can also assume that those delegates will involve a closure to capture some of the data. In order to limit the size of that closure, improve property access and caching, we create local variables holding values of the properties from the JsonPropertyInfo instance that we’re going to use within our get delegate.
// Encrypt on serialization
jsonPropertyInfo.Get = obj =>
{
var value = propertyGet(obj);
if (shouldSerialize is not null && !shouldSerialize(obj, value))
return value;
if (value is not string plainTextValue)
return isGetNullable ? null : string.Empty;
return Encrypt(plainTextValue);
};
Func<object, object?>. The input is the instance of the type being serialised, and the return value is used to set the property and may be null. Our function first needs to access the current value of the property, so we invoke the existing get delegate, passing in the object instance. This will return the plaintext value held in the property.ShouldSerialize will return null. When it is not null, we invoke the function, and if it returns false, skip encryption because this property will not be serialised in the final JSON. We have to return some value for the property getter, so we just return the original (plaintext) value.IsGetNullable property. This identifies whether the property accepts null values. If so, we can set return null; otherwise, we just return an empty string.Encrypt local function, passing it the plaintext value. We return the string it produces, which ensures that during serialisation, the encrypted value will be returned when System.Text.Json invokes this get delegate. It is that value which will then be serialised.Set delegate, used during deserialisation:
// Decrypt on deserialization
jsonPropertyInfo.Set = (obj, value) =>
{
string? plainTextValue = null;
if (value is string cipherText)
{
plainTextValue = Decrypt(cipherText);
propertySet(obj, plainTextValue);
}
};
Action<object, object?>, where the first object is the instance of the type being deserialised and the second nullable object is the value present in the JSON.jsonPropertyInfo.IsSetNullable and jsonPropertyInfo.IsRequired, but as far as my initial testing showed, this isn’t really necessary. If the property is required, an earlier check throws an exception when the JSON data doesn’t match a value for the property. In my scenario, where a property is non-nullable, if we don’t have a value, then we have nothing to set. Ultimately, this ends up set to null in that scenario, but since I’ll control both serialisation and deserialisation of data, I can ensure that won’t be the case. In other scenarios, you may want to explicitly handle IsSetNullable.
builder.Services.AddSingleton<EncryptedJsonTypeInfoResolverModifier>();
builder.Services.AddSingleton(sp =>
{
var modifier = sp.GetRequiredService<EncryptedJsonTypeInfoResolverModifier>();
return new JsonSerializerOptions
{
TypeInfoResolver = new DefaultJsonTypeInfoResolver
{
Modifiers = { modifier.EncryptStringsModifier }
}
};
});
EncryptedJsonTypeInfoResolverModifier. This is safe to do as it is stateless. In the future, this means we can inject dependencies into our modifier.JsonSerializerOptions singleton. This uses an implementation factory, which defines how to create the instance when it is first required. This allows us to resolve dependencies we need from the DI container. Here, we request the EncryptedJsonTypeInfoResolverModifier instance from the service provider. We can the new up the JsonSerializerOptions. We need to specify the TypeInfoResolver so that we can apply our modifications. We use a new DefaultJsonTypeInfoResolver and add to its list of modifiers our EncryptStringsModifier method, which matches the required Action<JsonTypeInfo> signature.
POST {{JsonEncryptionBlogExample_HostAddress}}
Accept: application/json
Content-Type: application/json
{
"accessToken": "AccessTokenValue"
}
info: Program[0]
Encrypted JSON: {"AccessToken":"AccessTokenValue_encrypted"}
info: Program[0]
Decrypted access token: AccessTokenValue
Conclusion
Have you enjoyed this post and found it useful? If so, please consider supporting me:
