ASP.NET Core Identity Token Providers – Under the Hood Part 2: Introducing Token Providers

Next up for my series on ASP.NET Core Identity I was interested in how the Identity library provides a way to create tokens which validate actions such as when a user first registers and we need to confirm their email address. This post took a lot longer to write than I expected it to as there are a lot of potential areas to cover. In the interests of making it reasonably digestible I’ve decided to introduce tokens and specifically look at the registration email confirmation token flow in this post. Other posts may follow as I dig deeper into the code and it’s uses.

As with part 1 let me prefix this post with two important notes.

  1. I am not a security expert. This series of posts records my own dive into the ASP.NET Identity Core code, publically available on GitHub which I’ve done for my own self-interest to try and understand how it works and what is available to me as a developer. Do not assume everything I have interpreted to be 100% accurate or any code samples as suitable production code.
  2. This is written whilst reviewing source mostly from the 3.0.0-rc1 release tag. I may stray into more recent dev code if implementations have changed considerably, but will try to highlight when I do so. One very important point here is that at the time of writing this post Microsoft have announced a renaming strategy for ASP.NET 5. Due to the brand new codebase this is now being called ASP.NET Core 1.0 and the underlying .NET Core will be .NET Core 1.0. This is going to result in namespace changes. I’ve used the anticipated new namespaces here (and will update if things change again).

What are tokens and why do we need them?

Tokens are something that an application or service can issue to a user and which they can later hand back as a way to prove their identity and often their authorisation for an action. We can use tokens in various places where we need to provide a mechanism to confirm something about them, such as that a phone number or email address actually belongs to them. They can also be used in other ways; Slack for example uses tokens to provide a magic sign in link on mobile devices.

Because of these potential uses it’s very important that they be secure and trustworthy since they could present a security hole into your application if used incorrectly. Mechanisms need to be in place to expire old or used tokens to prevent someone else using them should they gain access to them. ASP.NET Identity Core provides some basic tokens via token providers for common tasks. These are used by the default ASP.NET Web Application MVC template for some of the account and user management tasks on the AccountController and ManageController.

Now that I’ve explained what a token is let’s look at how we generate one.

Token providers

To get a token or validate one we use a token provider. ASP.NET Core Identity defines an IUserTokenProvider interface which any token providers should implement. This interface has been kept very simple and defines three methods:

Task<string> GenerateAsync(string purpose, UserManager<TUser> manager, TUser user);

This method will generate a token for a given purpose and user. The token is returned as a string.

Task<bool> ValidateAsync(string purpose, string token, UserManager<TUser> manager, TUser user);

This method will validate a token from a user. It will return true or false, indicating whether the token is valid or not.

Task<bool> CanGenerateTwoFactorTokenAsync(UserManager<TUser> manager, TUser user);

This indicates whether the token from this provider can be used for two factor authentication.

You can register as many token providers into your project as necessary to support your requirements. By default IdentityBuilder has a method AddDefaultTokenProviders() which you can chain onto your AddIdentity call from the startup file in your project. This will register the 3 default providers as per the code below. Token providers need to be registered with the DI container so they can be injected when required.

public virtual IdentityBuilder AddDefaultTokenProviders()
public virtual IdentityBuilder AddDefaultTokenProviders()
	var dataProtectionProviderType = typeof(DataProtectorTokenProvider<>).MakeGenericType(UserType);
	var phoneNumberProviderType = typeof(PhoneNumberTokenProvider<>).MakeGenericType(UserType);
	var emailTokenProviderType = typeof(EmailTokenProvider<>).MakeGenericType(UserType);
	return AddTokenProvider(TokenOptions.DefaultProvider, dataProtectionProviderType)
		.AddTokenProvider(TokenOptions.DefaultEmailProvider, emailTokenProviderType)
		.AddTokenProvider(TokenOptions.DefaultPhoneProvider, phoneNumberProviderType);

This code makes use of the TokenOptions class which defines a few common provider names and maintains a dictionary of the available providers, the key of which is the provider name. The value is the type for the provider being registered. The code for AddTokenProvider is as follows.

public virtual IdentityBuilder AddTokenProvider(string providerName, Type provider)
	if (!typeof(IUserTokenProvider<>).MakeGenericType(UserType).GetTypeInfo().IsAssignableFrom(provider.GetTypeInfo()))
		throw new InvalidOperationException(Resources.FormatInvalidManagerType(provider.Name, "IUserTokenProvider", UserType.Name));
	Services.Configure<IdentityOptions>(options =>
		options.Tokens.ProviderMap[providerName] = new TokenProviderDescriptor(provider);
	return this; 

Here you can see that the provider is being added into the ProviderMap dictionary and then registered with the DI container.

Registration email confirmation

Now that I’ve covered what tokens are and how they are registered I think the best thing to do is to take a look at a token being generated and validated. I’ve chosen to step through the process which creates an email confirmation token. The user is sent a link to the ConfirmEmail action which includes the userid and their token as querystring parameters. The user is required to click the link which will then validate the token and then mark their email as confirmed.

Validating the email this way is good practice as it prevents people from registering with or adding mailboxes which do not belong to them. By sending a link to the email address requiring an action from the user before the email is activated, only the true owner of the mailbox can access the link and click it to confirm that they did indeed signup for the account. We are trusting the user’s action based on something secure we have provided to them. Because the tokens are encrypted they are protected against forgery.

Generating the token

ASP.NET Core Identity provides the classes necessary to generate the token to be issued to the user in their link. The actual use of the Identity system to request the token and to include it in the link is managed by the MVC site itself, calling into the Identity API as necessary.

In ASP.NET MVC projects the generation of the confirmation email is optional and it is not enabled by default. However the code is there, but commented out within the AccountController. The UserManager class within Identity provides all the methods needed to call for the generation of a token and to validate it again later on. Once we have the token back from the Identity library we are then able to use that token when we send our activation email.

In our example we can call GenerateEmailConfirmationTokenAsync(TUser user). We pass in the user for which the token will be generated.

public virtual Task<string> GenerateEmailConfirmationTokenAsync(TUser user)
	return GenerateUserTokenAsync(user, Options.Tokens.EmailConfirmationTokenProvider, ConfirmEmailTokenPurpose);

GenerateUserTokenAsync requires the user, the name of the token provider to use (pulled from the Identity options) and the purpose for the token as a string. The ConfirmEmailTokenPurpose is a constant string defining the wording to use. In this case it is “EmailConfirmation”.

Each token is expected to carry a purpose so that they can be tied very closely to a specific action within your system. A token for one action would not be valid for another.

public virtual Task<string> GenerateUserTokenAsync(TUser user, string tokenProvider, string purpose)
	if (user == null)
		throw new ArgumentNullException("user");
	if (tokenProvider == null)
		throw new ArgumentNullException(nameof(tokenProvider));
	if (!_tokenProviders.ContainsKey(tokenProvider))
		throw new NotSupportedException(string.Format(CultureInfo.CurrentCulture, Resources.NoTokenProvider, tokenProvider));

	return _tokenProviders[tokenProvider].GenerateAsync(purpose, this, user);

After the usual null checks what this boils down to is checking through the dictionary of token providers available to the UserManager based on the tokenProvider parameter passed into the method. Once the provider is found it’s GenerateAsync method is called.

At the moment all of the three default TokenOptions providers are set to use the default token provider so by default the DataProtectionTokenProvider is being called which has the following GenerateAsync method.

public class TokenOptions
	public static readonly string DefaultProvider = "Default";
	public static readonly string DefaultEmailProvider = "Email";
	public static readonly string DefaultPhoneProvider = "Phone";

	public Dictionary<string, TokenProviderDescriptor> ProviderMap { get; set; } = new Dictionary<string, TokenProviderDescriptor>();

	public string EmailConfirmationTokenProvider { get; set; } = DefaultProvider;
	public string PasswordResetTokenProvider { get; set; } = DefaultProvider;
	public string ChangeEmailTokenProvider { get; set; } = DefaultProvider;

NOTE: This setup is slightly confusing as it appears that certain token providers, although registered would never be called based on the way the options are setup by default. This could be modified by changing the options and I have tried to query why this is setup this way by default.

For now though let’s look at the GenerateAsync method on the DataProtectionTokenProvider.

public virtual async Task<string> GenerateAsync(string purpose, UserManager<TUser> manager, TUser user)
	if (user == null)
		throw new ArgumentNullException(nameof(user));
	var ms = new MemoryStream();
	var userId = await manager.GetUserIdAsync(user);
	using (var writer = ms.CreateWriter())
		writer.Write(purpose ?? "");
		string stamp = null;
		if (manager.SupportsUserSecurityStamp)
			stamp = await manager.GetSecurityStampAsync(user);
		writer.Write(stamp ?? "");
	var protectedBytes = Protector.Protect(ms.ToArray());
	return Convert.ToBase64String(protectedBytes);

This method uses a memory stream to build up a byte array with the following elements:

  1. The current UTC time (converted to ticks within the extension method)
  2. The user id
  3. The purpose if not null
  4. The user security stamp if supported by the current user manager. The security stamp is a Guid stored in the database against the user. It gets updated when certain actions take place within the Identity UserManager class and provides a way to invalidate old tokens when an account has changed. The security stamp is changed for example when we change the username or email address of a user. By changing the stamp we prevent the same token being used to confirm the email again since the security stamp within the token will no longer match the user’s current security stamp.

These are then passed to the Protect method on an injected IDataProtector. For this post, going into detail about data protectors will be a bit deep and take me quite far off track. I do plan to look at them more in the future but for now it’s sufficient to say that the data protection library defines a cryptographic API for protecting data. Identity leverages this API from its token providers to encrypt and decrypt the tokens it has generated.

Once the protected bytes are returned they are base64 encoded and returned.

Validating the token

Once the user clicks on the link in their confirmation email it will take them to the ConfirmEmail action in the AccountController. That action takes in the userid and the code (protected token) from the link. This action will then call the ConfirmEmailAsync method on the UserManager which in turn will call a VerifyUserTokenAsync method. This method will get to appropriate token provider from the ProviderMap and call the ValidateAsync method.

Let’s step through the code which validates the token on the DataProtectorTokenProvider.

public virtual async Task<bool> ValidateAsync(string purpose, string token, UserManager<TUser> manager, TUser user)
		var unprotectedData = Protector.Unprotect(Convert.FromBase64String(token));
		var ms = new MemoryStream(unprotectedData);
		using (var reader = ms.CreateReader())
			var creationTime = reader.ReadDateTimeOffset();
			var expirationTime = creationTime + Options.TokenLifespan;
			if (expirationTime < DateTimeOffset.UtcNow)
				return false;

			var userId = reader.ReadString();
			var actualUserId = await manager.GetUserIdAsync(user);
			if (userId != actualUserId)
				return false;
			var purp = reader.ReadString();
			if (!string.Equals(purp, purpose))
				return false;
			var stamp = reader.ReadString();
			if (reader.PeekChar() != -1)
				return false;

			if (manager.SupportsUserSecurityStamp)
				return stamp == await manager.GetSecurityStampAsync(user);
			return stamp == "";
	// ReSharper disable once EmptyGeneralCatchClause
		// Do not leak exception
	return false;

The token is first converted from the base64 string representation to a byte array. It is then passed to the IDataProtector to be decrypted. Once again the details of how this works are too detailed for this post. The decrypted contents are passed into a new memory stream to be read.

The creation time is first read out from the start of the token. The expiration time is calculated by taking the token creation time and adding the token lifespan defined in the DataProtectionTokenProviderOptions. By default this is set at 1 day. If the token has expired then the method returns false since it is no longer considered a valid token.

It then reads the userId string and compares it to the id of the user (this is based on the userId from the link they get sent in their email. The account controller first uses that id to load up a user from the database. This ensures that the token belongs to the user who is attempting to use it.

It next reads the purpose and checks that it matches the purpose for the validation that is occurring (this will be passed into the method by the caller). This ensures a token is valid against only a specific function.

It then reads in the security stamp and stores it in a local variable for use in a few moments.

It then calls PeekChar which tries to get (but not advance) the next character from the token. Since we should be at the end of the stream here it checks for -1 which indicates no more characters are available. Any other value indicates that this token has extra data and is therefore not valid.

Finally, if security stamps are supported by the current user manager the security stamp for the user is retrieved from the user store and compared to the stamp it read from the token. Assuming they match then we can now confirm that the token is indeed valid and return that response to the caller.

Other token providers

In addition to the DataProtectionTokenProvider there are other providers defined within the Identity namespace. As far as I can tell these are not yet used based on the way the options are setup. I have actually queried this in an issue on the Identity repo. It may still be an interesting exercise for me to dig into how they work and differ from the DataProtectionTokenProvider. There is also the concept of an SMS verification token in the default ManageController for a default MVC application which doesn’t use a token provider directly

It would also be quite simple to implement your own token provider if you need to implement some additional functionality or store additional data within the token.

ASP.NET Identity Core 1.0.0 (Under the Hood) Part 1. Where did my salt column go?

Let me prefix this post / series of blog posts with two important notes.

  1. I am not a security expert. This series of posts records my own dive into the ASP.NET Identity Core code, publicly available on GitHub which I’ve done for my own self-interest to try and understand how it works and what is available to me as a developer. Do not assume everything I have interpreted to be 100% accurate or any code samples as suitable copy and paste production code.
  2. This is written whilst reviewing source mostly from the 3.0.0-rc1 release tag. I may stray into more recent dev code if implementations have changed considerably, but will try to highlight when I do so. One very important point here is that at the time of writing this post Microsoft have announced a renaming strategy for ASP.NET 5. Due to the brand new codebase this is now being called ASP.NET Core 1.0 and the underlying .NET Core will be .Net Core 1.0. This is going to result in namespace changes. I’ve used the anticipated new namespaces here (and will update if things change again).

Goals of this post / series

My personal reasons for digging into this code in the first place are simply to learn and further my own understanding. I’ll be investigating the sections that interest me and summarising as I do. I’m not sure where my investigations will take me so this may one post or many. I’m blogging about my findings as I hope they may be of interest to others wanting to know more about what happens when using this library in your projects. I’m going to try to edit this together in a readable fashion but I do expect it may be a little jumpy as I focus on the parts of the code I personally wanted to know more about. If you’re reading this blog post and see errors (and I expect there will be some) then please comment or contact me so I can update as appropriate.

Password Basics

I feel that before I go further I should try to explain a few key terms and concepts around password security. There are many more detailed resources which discuss security that go deeper into this but this primer should help when reading the rest of the post. If you know all of this then you may want to skip down to the next section below where I’ll start looking at the ASP.NET Identity code specifically.

With any login system, at a minimum we need to store two key pieces of information. A username and a password. The username uniquely identifies the user within your application and the password is a piece of information that only they should hold, used to verify to the application that they are, who they claim to be. This is the concept of authentication within a software application. In order for our applications to work with users we need to store these pieces of information somewhere so that when the user provides them, the application can look them up and compare them. Very commonly this will involve a database store and in ASP.NET that will likely be an SQL database table.

A simple solution would be to store the username as a string value (VARCHAR) and perhaps do the same for the password. This meets the initial requirement as we can now compare these against what a user provides us and if they match, open the door to the application. But there is a lot more to be considered and the most important element is securing that password from malicious intent. The problem being faced is how to store the password in such a way that it’s not plainly readable to anyone who has, or gains access to the database in which it has been stored. It’s well accepted that people will take the path of least complexity with passwords and often use the same password in many places. They will also likely use the same username where they can (especially if that username is their email address). So the risks go way beyond just your application should a password be compromised. It could well open the door to many other places where your user has also registered.

While some might assume that the database is secure and provides all the safety one needs for their user’s data, recent and numerous data exposures have proven that there are many people out there who can probably get to the database if they want to. I’m not planning to dive into the ways and means of that now but if you want more info check out posts from the likes of Troy Hunt who cover some other security aspects to consider. Even if the database is not compromised storing the passwords in plain text would still be a very poor decision since your internal staff also should not have access to them. A password should be as a user expects and be a secret to themselves only. They should be able to expect that you will take due care and responsibility when working with such a piece of data.

So we now want to store the password in a form that is secure from anyone who is looking at the database table, but we also need to be able to verify the user at a later time by comparing their input, to the password stored for their account. A solution could be reversible encryption where we encrypt the password data when we save it to the database and then decrypt it later to compare it with user input. However that also has concerns and issues, again slightly outside the scope of my intentions for this blog post. The short version being that anything reversible still presents a fair degree of insecurity when it comes to passwords.

This is where the concept of a one way hash comes into the picture. A hash is a mathematically repeatable process whereby we can take the password, put it through hashing and then store that data, instead of a plain text password. The word “repeatable” is important here. The reason hashing works is that if you provide the same text and put it through the same hashing process, the result will always be the same. At login we take the provided password, hash it (using the same algorithm used when the password was first saved to the database) and then compare the hashed value against the stored value. If they match, we have authenticated the user.

The advantages are that if the database is compromised, the hashed value is more secure than plain text. The hash cannot be reversed to generate the original plain text password either. But we’re still not done. With the advent of faster processing, even hashes are breakable. For a start, users commonly use simple and guessable passwords. Attackers can take advantage of that by pre-calculating hash values for common passwords, using the same algorithms that an application might use to create a mapping between potential passwords and hashed passwords. They could then use this against a compromised database and quickly work out many of the passwords being used.

We therefore must again go a step further and try to secure against this. So finally we come to the concept of a salt which adds more complexity and uniqueness to the passwords being hashed. Think of an example where two users choose to use the same exact password for your system. Those passwords when hashed will be identical in the database. This could be valuable information to hackers. Therefore to make things more unique, we use a random set of characters, called a salt. This salt is added to the password so that we get a unique password and therefore unique hash for each user, even if they use the same passwords. This mainly helps defend from the dictionary based attacks as hackers would have to calculate the lookup table once per user with their salt.

Finally we have hashing iterations. Hashing once is a start but with modern computer power it’s pretty fast to generate a lookup table. Iterations help somewhat by making the process more computationally expensive and time consuming. The aim of which is to make hacking a password database too much work vs the potential reward.

ASP.NET Identity and Password Hashing

In Identity 2.0 the hashed password and the salt were stored in separate columns in the database. It was only when I looked at a database for a new ASP.NET Core website that I noticed that there was no salt column and I wondered what was happening (hence this blog post)!

So to understand let’s follow the flow of a registering user with a focus on the password so we can understand what ASP.NET Core Identity 1.0.0 is doing to securely save it to the database (and answer my question – where did the salt column go!)

We start in the AccountController with a call to CreateAsync in the Microsoft.AspNet.Identity.UserManager class passing in an IdentityUser object and the password to be hashed and saved.

After running through any password validators we call on an instance an  IPasswordHasher<TUser> (which is injected into the UserManager when it is first constructed).

The PasswordHasher is constructed with an instance of PasswordHasherOptions. The class PasswordHasherOptions defines the settings used for the password hashing. By default the compatibility mode is set to Identity v3 mode, but if you needed to support V2 and below this can be set in the options. There is also a setting for the iterations to use when creating the hash which by default is set to 10,000. Finally this class defines and creates a static instance of the RandomNumberGenerator object found in the mscorelib.

UserManager.CreateAsync calls the HashPassword method.

The HashPassword method checks the compatibility mode in the options and flows through the appropriate method to handle either V2 and V3 identity. We’ll look at the V3 flow here using HashPasswordV3.

Here is the code…

private byte[] HashPasswordV3(string password, RandomNumberGenerator rng)
    return HashPasswordV3(password, rng,
    prf: KeyDerivationPrf.HMACSHA256,
    iterCount: _iterCount,
    saltSize: 128 / 8,
    numBytesRequested: 256 / 8);

private static byte[] HashPasswordV3(string password, RandomNumberGenerator rng, KeyDerivationPrf prf, int iterCount, int saltSize, int numBytesRequested)
    // Produce a version 3 (see comment above) text hash.
    byte[] salt = new byte[saltSize];
    byte[] subkey = KeyDerivation.Pbkdf2(password, salt, prf, iterCount, numBytesRequested);

    var outputBytes = new byte[13 + salt.Length + subkey.Length];
    outputBytes[0] = 0x01; // format marker
    WriteNetworkByteOrder(outputBytes, 1, (uint)prf);
    WriteNetworkByteOrder(outputBytes, 5, (uint)iterCount);
    WriteNetworkByteOrder(outputBytes, 9, (uint)saltSize);
    Buffer.BlockCopy(salt, 0, outputBytes, 13, salt.Length);
    Buffer.BlockCopy(subkey, 0, outputBytes, 13 + saltSize, subkey.Length);
    return outputBytes;

Breaking down the parameters of the main overloaded method:

  1. A string representing the password to be hashed
  2. An instance of a RandomNumberGenerator (defined in the mscorelib assembly). I’m not going to delve into how that code works here but the CoreCLR project on GitHub contains the code if you want to take a look for yourself.
  3. A choice of the PRF (pseudorandom function family) to use for the key derivation. In this case HMACSHA256 is the standard for Identity V3
  4. An integer representing the iteration count to use when hashing (coming from the PasswordHasherOptions)
  5. The size (number of bytes) to be used for the salt – 128 bits / 8 to calculate the byte size of 16
  6. The number of bytes for the hashed password – 256 bits / 8 to calculate the byte size of 32

A new byte array is created to hold the 16 byte salt which is generated with a call to GetBytes(salt) on the RandomNumberGenerator. This populates the byte array with 16 random bytes. Then a hashed password is created using PBKDF2 (Password-Based Key Derivation Function 2) key derivation, taking the password, the salt, the PRF of HMACSHA256, the number of iterations to use and the number of bytes required as the length of the hashed password.

We now have a hashed password and the random salt used to generate that password. In earlier versions of identity I saw these being saved into two unique columns inside the database. However with Identity 3.0 / ASP.NET Core Identity 1.0 there is only the PasswordHash column. To save the password and hash a new byte array is created. It is the size of the salt + hashed password + 13 extra bytes used to store some meta data about the hashing which took place. These form a single byte array which will be base64 encoded as a string to save into the database.

Construction of the Final Byte Array

The first byte is a format marker and for Identity 3.0 / ASP.NET Core 1.0.0 this is set to 1 (defined in hex).

The next byte stores the PRF that was used. This is the value of the enum available in Microsoft.AspNetCore.Cryptography.KeyDerivation. This belongs in the ASP.NET DataProtection assembly (source on github)

The next 4 bytes store the iteration count used to generate the hash

The next 4 bytes store the salt size

Notice with each of these items there is a call to WriteNetworkByteOrder, a helper method within the PasswordHasher.

private static void WriteNetworkByteOrder(byte[] buffer, int offset, uint value)
    buffer[offset + 0] = (byte)(value >> 24);
    buffer[offset + 1] = (byte)(value >> 16);
    buffer[offset + 2] = (byte)(value >> 8);
    buffer[offset + 3] = (byte)(value >> 0);

What this is doing in short is splitting the 32 bit integer into the component bytes using bitwise shifting and then storing those in big-endian (network byte) order into the array.

HashPasswordV3 then copies in the bytes from the salt and the password hash to create the final byte array which is returned to the main HashPassword method. This is converted to a base64 encoded string. This is the string is then written to the to the database.

And there we have it. That’s a fairly deep dive of how passwords are hashed inside ASP.NET Core 1.0.0 identity and then stored in the database.

Introducing My Blog

I’ve been thinking about creating a blog on and off for a year or so. I like the idea of sharing what I learn and if nothing else it gives me somewhere to write stuff down that I might later need. I’ve had a blog before and finding the time and discipline to keep it going was tough. I hope I do better this time around.

It’s mostly going to be about ASP.NET and C# since those are what I work with day to day. It’s an exciting time with the nearing release of ASP.NET Core 1.0 and I’ll be mostly investigating the code and concepts for that product as I use it for new projects. I enjoy learning new things and I want to get as good a grip as possible on how things work and best ways to use this technology. I’ll cover the things I learn as I go.

As far as intro’s go, I think that’ll do. We’ll see how well I keep this up and if I find the time to post.