Passwords are like underwear – Keep them hidden using Azure MSI

Introduction

We keep the credentials and other secrets of  the application in the source files. These secrets are visible to developers and being pushed to the source control. In order to avoid this, we can keep the secrets in centralized key management systems – but the credentials of the centralized key management system should be kept in the source files, resulting that at least one credential being placed in the source files.

x.jpeg

Azure Managed Service Identity (MSI) solves this bootstrap problem. It eliminates the need of storing any credentials in the source files.

MSI is supported by different Azure services. This post explains how to set up MSI in the Azure Web App and retrieve a secret stored in Azure Key Vault (a database password).

Setting MSI in Azure Web App

MSI works on Azure AD authentication.  When MSI is enabled in a service (web app) it creates an application principle in the same tenant where the subscription of the Web App is attached to.

Navigate to your Web App, select the Managed Service Identity option and switch the feature ON.

enable MSI in azure web apps

This will create an application principle in the AAD with the same name of the Web App (teslaspace).

2

You can also notice the following section with the specific IDs in the ARM template of Web App.

3

Creating Azure Key Vault and set up Developer Access for Secrets

Now we should create a Azure Key Vault in a subscription attached to the same tenant. We will store our credentials and other sensitive strings in this Key Vault.

Creating a Azure Key Vault is straight forward and it can easily be done via the portal. I have skipped those steps in this post. Read more articles about Azure Key Vault

Once the Key Vault is created, we should add the Web App Service Principle to the Key Vault access policies. Give minimum permissions are possible.

4

This principle is given only Get permissions to the secret. This is enough to retrieve a sensitive string from Azure Key Vault.

We should create a secret in the Azure Key Vault and obtain the secret URI.

5

Navigate to the secret and copy the URI. Secret URIs are in the below format

https://{key vault name}.vault.azure.net/secrets/{secret name}/{version no}

Implementation

We have done below items.

  1. We created Web App and enabled MSI
  2. This created a service principle in the corresponding tenant
  3. We created a Azure Key vault
  4. We granted some permissions to the Web App Service Principle
  5. We created a secret in Key Vault and obtained the URI

Now, the MSI enabled Web App should connect to the Azure Key Vault and obtain the value of the secret. In order to connect to the Key Vault it does not require any credentials to be stored in the source code.

There is a NuGet package which facilitates this process.

Install-Package Microsoft.Azure.Services.AppAuthentication -Version 1.0.0-preview

Also, add a reference to Azure Key Vault

Install-Package Microsoft.Azure.KeyVault -Version 2.3.2

The below code will retrieve the secret for you, ex – a db password


public async Task<string> RetrieveSecretBasedOnMSIAsync()
 {
 AzureServiceTokenProvider azureServiceTokenProvider = new AzureServiceTokenProvider();
 string accessToken = await azureServiceTokenProvider.GetAccessTokenAsync("https://management.azure.com/");

 var kv = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
 var secret = await kv.GetSecretAsync("secret uri");

return secret.Value;
 }

If you turn off the MSI from the Web App the above code will throw an exception that it cannot authenticate to the AAD.

Developer Environment

The above works in the production as MSI is enabled in the Web App, but how to set the same configuration in the developer machine and make sure that the authentication happens to AAD via the above code.

This is straight forward, via Azure CLI. You should install Azure CLI 2.0 or above in your development machine.

Also, we need a user principle to authenticate to the AAD. The same principle should be added to the Azure Key Vault with the required permissions (Get permissions for the secrets).

So create a user in AAD. ex – dev1@massroverdev.onmicrosoft.com. Add the user principle to the Key Vault as shown above. You don’t need to assign any subscription access to this user.

That’s all we should do in the Azure. In the developer machine, after installing the Azure CLI 2.0 or above, login to the Azure CLI using below command.

az login --allow-no-subscriptions

This will pop up the device login and complete the login using the credentials of the created user.

Note : You have to provide the — allow-no-subscriptions flag since user does not have any subscription level access. This is the bare minimum developer permission that can be given in MSI.

As long as the developer is logged in to the Azure CLI which has the required permissions in the Azure Key Vault, the above code will work.

Things to note

  • MSI service principle is a AAD V1 approach. V2 or MSAL support is yet to be released. So if you’ve a V2 app registration, enabling MSI will create another V1 application in the tenant. Read more
  • Developer gets to see the secrets during debug. There are workarounds to eliminate this.
    • Use a different KV in production environment
    • User a token based access for the service which have AAD authentication. This is possible in SQL Databases. But it adds additional complexity in handling tokens expiry and etc. Read more

 

 

 

Advertisement

Azure Key Vault Logging

This post goes with the series of my posts under the Azure Key Vault.

I assume that you know about Azure Key Vault and have used it, and continue this article. If you’re new to Azure Key Vault, please review the below links.

You can read more about Azure Key Vault and how to use it from this post.

PowerShell script to provision the Key Vault and the C#.NET sample to use it in the GitHub.

An Open source tool to manage Key Vault: Azure Key Vault Manager

Enabling Logging Diagnostics for Azure Key Vault

Recently Azure Key Vault team has announced the logging feature for the Key Vault (which is one of the highly required features).

Logs are written to a storage account in the Azure. So first create a storage account. Then in the PowerShell execute the following commands. Assuming that you have a vault and storage account.

It is good keep the storage account in the same Resource Group of the Key Vault as management would be easy.



We have the vault and storage details in variables, now time to setup the diagnostics


Viewing Logs

Logs are saved as JSON documents in the blob storage of the provided storage account. Do some activities which perform some operations in the Key Vault and get the JSON.

The below is log snippet for retrieving the vault. Note the operation name as VaultGet also the log provides information like the duration and client IP addresses. In the identity section it also provides the used identity information (the Azure Active Directory Identity name) for the specified operation.

The below is another JSON document snippet for the SecretGet operation. Along with the other information the request Uri property gives the details of which secret and the version information.

Disabling the logging Diagnostics

Execute the following line to disable the logging. (assuming the $vault and $storage variables are set as shown above)


Azure Key Vault Manager

Azure Key Vault is generally available. If you use Azure Key Vault in your projects, then there’s a high probability that you felt the need of a handy dev tool to manage your Vault.

Here it is. http://keyvaultmanager.azurewebsites.net

GitHub: https://github.com/thuru/AzureKeyVaultManager

More about Azure Key Vault: https://thuruinhttp.wordpress.com/2015/05/30/azure-key-vault-setup-and-usage-scenarios/

Azure Key Vault setup and usage scenarios

Introduction

At the time of this writing Azure Key Vault is in preview.  Azure Key Vault is a secure store solution for storing string based confidential information.

The reason I’ve mentioned that the string based confidential information is that you can store a key used for encrypting a file, but you cannot store the encrypted file itself as a file object; because some people have the confusion what could be stored inside the Key Vault.

Azure Key Vault – http://azure.microsoft.com/en-gb/services/key-vault/

Key Vault store 2 types of information

  1. Keys
  2. Secrets

Secrets– This could be any sequence of byte under 10 KB. Secrets can be retrieved back from the vault. Very much suitable for retrievable sensitive information like connection strings, passwords and ect. From the design point of view, we can either retrieve the keys every time we need or retrieve it once and store in the cache.

Keys – Keys could be imported to the vault from your existing vaults, also if your organization has Hardware Security Modules (HSM) you can directly transfer them to HSM based Azure Key Vault. Keys cannot be retrieved from the vault. For example if you store the key of a symmetric encryption which encrypts the files, you should send the files to vault ask the vault to encrypt / decrypt the data. Since keys cannot be retrieved from the vault this provides a higher isolation.

Keys could be stored in 2 different ways in the vault

  1. Software protected keys
  2. Hardware protected keys

Software Protected Keys – This is available in the standard tier of the vault. Compared to the Hardware protection this is theoretically less secured.

Hardware Protected Keys – HSMs are used to add premium hardware based circuitry secure storage for the keys. The most advanced key vault system available.

 

Provisioning Azure Key Vault

As Azure Key Vault is used to store sensitive information the authentication to the Azure Key Vault should happen via Azure AD. Let me explain it in simple steps.

  1. First a subscription administrator (either the service admin or co-admin) will create a Azure Key Vault using PowerShell.
  2. Then the admin registers an Azure AD application and generate the App Id and the App Secret Key.
  3. Admin grants the permission (trust) to the App to access the Key Vault using PowerShell.
  4. The subscription where the Vault is created should be attached to the Azure AD where the accessing app in the above step is created.
  5. This ensures that accessing app is an object of the Azure AD on which the subscription where the Vault is created is attached to.

Sometimes the 4th and 5th points might be bit confusing and you might face them especially when dealing with the multiple Azure subscriptions. See the below image for a clear picture.

Picture5

Assume that you have two subscriptions in your Azure Account, if you create the Vault in the Development subscription the app which can authenticate to the Vault should be in the Default AD. If you want to have the app in the Development AD you have to change the directory of the Development subscription.

Usage

Assume MassRover is a fictional multi tenant application on Azure.

ISV owns the Azure Key Vault

Scenario 1 (using secrets to for the encryption) – MassRover allows users to upload documents and it promises high confidential data security to its tenants. So it should encrypt the data at rest. MassRover uses it’s own Azure Key Vault to store the secrets (which are the encryption keys).  A Trust has been setup between the Azure Key Vault and MassRover AD client application. MassRover Web App authenticates to the Azure Key Vault retrieves the secrets and performs the encryption / decryption of the data.

Picture1

 

Scenario 2 (using keys) – MassRover Azure Key Vault stores the keys which cannot be retrieved out of the Vault. So the web app authenticate itself with the Vault and sends the data to the Vault to perform the encryption of decryption. This scenario has higher latency than scenario 1.

Picture2

 

Tenant owns the Azure Key Vault

Tenants can own their Key Vault and give access to MassRover by sharing the the authorized application Id and application secret. This is an added benefit if the tenants worry about ISVs keeping the keys in their subscription and administrative boundary. Tenant maintained Key Vaults give additional policy based security for sure but latency is high since data transfer has to happen across different wires. (this could be solved to certain extent if the tenant provisions the Key Vault in same region).

Tenant maintained Key Vault also has 2 scenarios explained above, as either to go with the secrets or go with the keys.

Scenario 3 (using secrets)

Picture3

Scenario 4 (using keys)

Picture1

 

Useful links

Azure Key Vault NuGet packages (at the time of this writing they in pre release stage : http://www.nuget.org/packages/Microsoft.Azure.KeyVault/

PowerShell for provisioning Azure Key Vault and .NET code sample : https://github.com/thuru/AzureKeyVaultSample

Channel 09 – http://channel9.msdn.com/Shows/Cloud+Cover/Episode-169-Azure-Key-Vault-with-Sumedh-Barde