Automate the use of secrets in the cloud using Azure Key Vault

Published in

medialesson

9 min readMar 22, 2023

E2E example on how to automate the deployment and use of Azure Key Vault using Bicep, RBAC and Azure Pipelines.

In a perfect model world we would not require a secret store like Azure Key Vault any more as all services would be authorizing between each other using managed identities and role base access control (RBAC). Until this materialized there are still a bunch of services in Azure that don’t support RBAC just yet and also sometimes there are secrets from outside of Azure like 3rd party api keys that need to be stored securely and made accessible to Azure services.

This example shows

how to create and configure a Key Vault and secrets using Bicep Infrastructure as Code (IaC) run from a Azure Pipeline.
how to authorize the use of secrets in a Key Vault to other Azure services like a Azure Functions app using system assigned managed identities and role based access control (RBAC) all from Bicep
how to use a secret in a Azure Functions app

Prepare the environment

So let’s get started by

Creating a new resource group as a container for our services
Creating a new service principal to use to authenticate with Azure from our deployment pipeline
Assign the required roles to the service principal

Make sure you have a subscription in Azure and also the Azure CLI installed on your machine.

Create a new resource group

az group create -n keyvaultdemo -l westeurope

Create a service principal

The Azure Pipeline will use this service principal in a service connection to authenticate with Azure for deployment operations. When creating a service principal it’s also possible to directly assign this principal a role for a given Azure resource:

az ad sp create-for-rbac \
  --name AzureDevOps-keyvaultdemo \
  --scopes /subscriptions/{YOUR_SUBSCRIPTION_ID}/resourceGroups/keyvaultdemo \
  --role Contributor

Take note of the appId, password and tenant as we will need this information later.

The app we will create later needs the permission to read secrets from a key vault. Therefore the Azure Pipeline needs the permission to assign roles to Azure resources. So we need to give this permission in form of a role to the service principal as this principal will be used by the pipeline to authenticate. To create an additional role assignment for a service principal we need to know it’s principalId. So let’s request this using the CLI:

az ad sp show \
  --id {APP_ID} \
  --query '{displayName: displayName, principalId: id}'

This will output the displayName and principalId. We can now use the principalId to assign the role “Role Based Access Control Administrator” to the service principal:

az role assignment create \
  --assignee "{PRINCIPAL_ID}" \
  --role "/subscriptions/{YOUR_SUBSCRIPTION_ID}/providers/Microsoft.Authorization/roleDefinitions/f58310d9-a9f6-439a-9e8d-f62e7b41a168" \
  --resource-group "keyvaultdemo"

If you’re wondering where this “magic” guid for the role definition is coming from. Here is a great place to look up Azure builtin roles: https://www.azadvertizer.net/azrolesadvertizer/f58310d9-a9f6-439a-9e8d-f62e7b41a168.html

Create a service connection in Azure DevOps

With the information we got from creating the service principal we can now setup a new service connection in a Azure DevOps project that can be used by a pipeline. So let’s head over to our project > Project Settings > Service Connections > New Service Connection and choose Azure Resource Manager and then Service principal (manual).

Sadly it’s a bit confusing because the fields here are named different than what we got from creating the service principal. So here is a mapping:

Service Principal Id → appId of the principal, NOT the principalId!
Service principal key → password of the principal
Tenant ID → tenant of the principal

Infrastructure as Code

Now that we have prepared the environment for our demo it’s time to start adding bicep templates to our repo to model the infrastructure. I like to use modules for different services to maintain overview of the system I’m building.

Create a Key Vault

The first module is the key vault itself. Here is a simple example:

param name string
param location string

resource keyVault 'Microsoft.KeyVault/vaults@2022-07-01' = {
  name: name
  location: location
  properties: {
    enabledForTemplateDeployment: true
    enableRbacAuthorization: true
    tenantId: subscription().tenantId
    sku: {
      family: 'A'
      name: 'standard'
    }
  }
}

output name string = keyVault.name

I like to use outputs as this allows to also set implicit dependencies between bicep modules.

Create a Secret

The sample secret shall also be deployed using bicep and will be a child resource of the key vault:

param keyVaultName string
param name string
@secure()
param value string

resource keyVault 'Microsoft.KeyVault/vaults@2022-07-01' existing = {
  name: keyVaultName
  resource secret 'secrets' = {
    name: name
    properties: {
      value: value
    }
  }
}

output secretUri string = keyVault::secret.properties.secretUri

Note here that the param value is marked as secure() to prevent the actual value of the secret being written into any log during deployment.

Since the secret is a seperate module the parent key vault is referenced using the existing keyword.

The output returns the secretUri to this secret that we will use as a link in the environment variables of the app later. Also note the :: syntax of access child resources.

Create a Functions app

To test if access to the key vault and the secret works we need to have a sample app. I’m going to use a Azure Function here but note that this works with pretty much any compute options in Azure like Azure App Service or Azure Container Apps:

param name string
param location string
param appSettings array = []

resource storageAccount 'Microsoft.Storage/storageAccounts@2021-08-01' = {
  name: name
  location: location
  sku: {
    name: 'Standard_LRS'
  }
  kind: 'Storage'
}

var storageConnectionString = 'DefaultEndpointsProtocol=https;AccountName=${name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${storageAccount.listKeys().keys[0].value}'

resource applicationInsights 'Microsoft.Insights/components@2020-02-02' = {
  name: name
  location: location
  kind: 'web'
  properties: {
    Application_Type: 'web'
    Request_Source: 'rest'
  }
}

resource hostingPlan 'Microsoft.Web/serverfarms@2021-03-01' = {
  name: name
  location: location
  kind: 'linux'
  sku: {
    name: 'Y1'
    tier: 'Dynamic'
  }
  properties: {
    reserved: true
  }
}

resource functionApp 'Microsoft.Web/sites@2021-03-01' = {
  name: name
  location: location
  kind: 'functionapp'
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    serverFarmId: hostingPlan.id
    siteConfig: {
      linuxFxVersion: 'DOTNET|6.0'
      appSettings: union([
        {
          name: 'AzureWebJobsStorage'
          value: storageConnectionString
        }
        {
          name: 'WEBSITE_CONTENTAZUREFILECONNECTIONSTRING'
          value: storageConnectionString
        }
        {
          name: 'WEBSITE_CONTENTSHARE'
          value: toLower(name)
        }
        {
          name: 'FUNCTIONS_EXTENSION_VERSION'
          value: '~4'
        }
        {
          name: 'APPINSIGHTS_INSTRUMENTATIONKEY'
          value: applicationInsights.properties.InstrumentationKey
        }
        {
          name: 'FUNCTIONS_WORKER_RUNTIME'
          value: 'dotnet'
        }
      ], appSettings)
      ftpsState: 'FtpsOnly'
      minTlsVersion: '1.2'
    }
    httpsOnly: true
  }
}

output name string = functionApp.name
output principalId string = functionApp.identity.principalId

The interesting bit here is that the module has an input parameter of an array of app settings that is merged with the app settings of the function app. This allows us to provide additional dynamic app settings like the secretUri when calling this module.

Also note that we enable the SystemAssigned identity on the function app. This is crucial as this identitiy will be used by the service to authenticate itself with the key vault.

Create a role assignment

To access the key vault the function app’s identity needs to be in an appropriate role to have the permission to do so:

param keyVaultName string
param principalId string 
@description('Defaults to Key Vault Secrets User')
param roleId string = '4633458b-17de-408a-b874-0445c86b69e6'

resource keyVault 'Microsoft.KeyVault/vaults@2022-07-01' existing = {
  name: keyVaultName
}

resource keyVaultAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(subscription().subscriptionId, keyVaultName, roleId, principalId)
  scope: keyVault
  properties: {
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', roleId)
    principalId: principalId
    principalType: 'ServicePrincipal'
  }
}

Putting it together

With all modules in place we can now put it all together in a main.bicep file:

param name string
param location string = resourceGroup().location
@secure()
param mySecretValue string

module keyVault 'services/keyvault.bicep' = {
  name: 'deploy-kv-${name}'
  params: {
    name: name
    location: location
  }
}

module mySecret 'services/keyvault-secret.bicep' = {
  name: 'deploy-kv-${name}-secret-mysecret'
  params: {
    keyVaultName: keyVault.outputs.name
    name: 'MySecret'
    value: mySecretValue
  }
}

module functionApp 'services/function-app.bicep' = {
  name: 'deploy-fa-${name}'
  params: {
    name: name
    location: location
    appSettings: [
      {
        name: 'MySecret'
        value: '@Microsoft.KeyVault(SecretUri=${mySecret.outputs.secretUri})'
      }
    ]
  }
}

module functionAppKeyVaultRoleAssigment 'services/keyvault-role-assignment.bicep' = {
  name: 'deploy-fa-kv-ra-${name}'
  params: {
    keyVaultName: keyVault.outputs.name
    principalId: functionApp.outputs.principalId
  }
}

By working with output in subsequent module calls we make sure the modules are deployed in the right order without having to specify explicit dependencies. Also note how the link to the sample secret is added as app settings to the function app with the key vault syntax.

Create sample app

To verify that the secret is available to the app let’s have a test output of the secret in a simple http function. Therefore we need to create a new function app either in Visual Studio oder Visual Studio Code and add a new http function:

public static class HelloWorldWithASecret
{
    [FunctionName("HelloWorldWithASecret")]
    public static IActionResult Run(
        [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = null)] HttpRequest req)
    {
        string mySecret = Environment.GetEnvironmentVariable("MySecret");
        return new OkObjectResult($"Hello World, my secret is {mySecret}");
    }
}

Of course this works not only with C# but any of the supported Functions languages as the secret is provided by the runtime as environment variable. Which is also why I prefer using the key vault with the link syntax over accessing it through code.

Create build and deployment pipeline

The final piece of the demo is defining a pipeline to build and deploy the infrastructure and build and deploy the function app.

First let’s define some variables that we will need accross the different pipeline jobs:

variables:
  AZ_RESOURCE_GROUP_NAME: keyvaultdemo
  AZ_ENVIRONMENT_NAME: keyvaultdemopb
  BICEP_ARTIFACT_NAME: bicep
  FUNCTION_APP_ARTIFACT_NAME: functionapp
  SERVICE_CONNECTION_NAME: AzureTest

Build the infrastructure

Next up we build the bicep files to make sure everything is correct in there and then publish the result:

jobs:
  - job: buildInfrastructure
    displayName: Build Infrastructure
    variables:
      STAGING_DIRECTORY: bicep
    steps:
      - script: az bicep build --file ./.bicep/main.bicep
        displayName: Build Bicep
      - task: CopyFiles@2
        displayName: 'Copy Bicep'
        inputs:
          SourceFolder: '.bicep'
          Contents: '**'
          TargetFolder: $(Build.ArtifactStagingDirectory)/$(STAGING_DIRECTORY)
      - task: PublishBuildArtifacts@1
        displayName: Publish Bicep
        inputs:
          PathtoPublish: $(Build.ArtifactStagingDirectory)/$(STAGING_DIRECTORY)
          ArtifactName: $(BICEP_ARTIFACT_NAME)
          publishLocation: Container

Deploy the infrastructure

The published infrastructure artifact can now be deployed in a deployment job:

  - deployment: deployInfrastructure
    displayName: Deploy Infrastructure
    environment: production
    dependsOn: buildInfrastructure
    strategy:
      runOnce:
        deploy:
          steps:
            - download: current
              artifact: ${{ variables.BICEP_ARTIFACT_NAME }}
            - task: AzureCLI@2
              displayName: Deploy Bicep
              inputs:
                azureSubscription: $(SERVICE_CONNECTION_NAME)
                scriptType: pscore
                scriptLocation: inlineScript
                inlineScript: az deployment group create -g $(AZ_RESOURCE_GROUP_NAME) --template-file $(Pipeline.Workspace)/$(BICEP_ARTIFACT_NAME)/main.bicep --parameters name='$(AZ_ENVIRONMENT_NAME)' mySecretValue='$(MY_SECRET)'

The variable MY_SECRET is not defined in the pipeline definition itself. We will add this later as pipeline secret.

Build the app

This is an example on how to build a function app in case we’re using C#/.NET:

  - job: buildFunctionApp
    displayName: Build Function App
    variables:
      STAGING_DIRECTORY: functionapp
    steps:
      - task: UseDotNet@2
        displayName: "Use Dotnet 6"
        inputs:
          packageType: 'sdk'
          version: '6.0.x'  
      - task: DotNetCoreCLI@2
        displayName: "Restore Function App"
        inputs:
          command: "restore"
          projects: "src/**/*.csproj"
      - task: DotNetCoreCLI@2
        displayName: "Build Function App"
        inputs:
          command: "build"
          projects: "src/**/*.csproj"
          arguments: --output $(Build.ArtifactStagingDirectory)/$(STAGING_DIRECTORY) --configuration Release
      - task: ArchiveFiles@2
        displayName: "Archive Functions files"
        inputs:
            rootFolderOrFile: $(Build.ArtifactStagingDirectory)/$(STAGING_DIRECTORY) 
            includeRootFolder: false
            archiveType: zip
            archiveFile: $(Build.ArtifactStagingDirectory)/$(STAGING_DIRECTORY)/$(Build.BuildId).zip
            replaceExistingArchive: true
      - publish: $(Build.ArtifactStagingDirectory)/$(STAGING_DIRECTORY)/$(Build.BuildId).zip
        displayName: "Publish Functions"
        artifact: $(FUNCTION_APP_ARTIFACT_NAME)

Deploy the app

And here is how we can deploy the function app to Azure:

  - deployment: deployFunctionApp
    displayName: Deploy Function App
    environment: production
    dependsOn: 
      - deployInfrastructure
      - buildFunctionApp
    strategy:
      runOnce:
        deploy:
          steps:
            - download: current
              artifact: ${{ variables.FUNCTION_APP_ARTIFACT_NAME }}
            - task: AzureFunctionApp@2
              displayName: "Deploy Function app"
              inputs:
                azureSubscription: $(SERVICE_CONNECTION_NAME)
                appType: functionAppLinux
                appName: $(AZ_ENVIRONMENT_NAME)
                package: "$(Pipeline.Workspace)/$(FUNCTION_APP_ARTIFACT_NAME)/$(Build.BuildId).zip"

The final step is to push all code to our repo and create a new pipeline out of this definition. In the pipeline we can now add a new variable MY_SECRET and mark it as secret:

Now let’s run the pipeline:

Note: sometimes the deploy function app job can fail on the first run as the newly create function app in Azure might not be ready yet. Don’t worry, just run it again in a minute and it should be fine.

Let’s also checkout the content of our resource group in Azure:

We can also verify if the function app is allowed to read secret in the key vault > Access control > Role assignments:

So everything is looking great, so let’s call out http function and see what our secret is:

Summary

We’ve learned that it’s possible to

define all infrastructure and configuration as code using bicep including secrets
use RBAC to authorize Azure services to access secrets in a key vault
access secrets through environment variables without writing code