0

My requirement is to automate the creation of secrets in Keyvault without exposing the values of these secrets during the pipeline execution(AzureDevops Server).

tried, different options such as having input paramaters (it will display the values when its perform initialization) , runtime variables etc (list of variables, or dynamic creation as per input is not possible), but couldn't fulfil our requirements.

Finally planned to use the below script to read values from library group and create the values in key vault. But when we are masking the secret values in Library group (by locking), the secrets are getting created with null value only. So is there any way to retrieve the values secret variables and create them in key vault only if the secret value is different(if existing)

-bash: | 
   az login --service-principal --username $(spid) --password $(spsecret) --tenant $(tenantid)
   az account set --subscription ${{ variables.subscription }}
   az config set extension.use_dynamic_install=yes_without_prompt
   groupID=`az pipelines variable-group list -p myproject --group-name ${{ environment }}-${{ parameters.myapp}}-kv-secret --query '[].id' -o tsv`
   echo "grouup id is $groupID"
   variables=$(az pipelines variable-group variable list -p myproject --group-id $groupID --output json)
   echo "$variables" | jq -r 'keys[] as $k | "\($k)=\(.[$k].value)"' | while read variable; do
   name=$(echo "$variable" | cut -d= -f1)
   value=$(echo "$variable" | cut -d= -f2)
   if ! az keyvault secret show --vault-name "$(${{ variables.podkv }})" --name "$name" &>/dev/null; then
     expiryDate=$(date -u -d '+2 years' '+%Y-%m-%dT%H:%MZ')
     az keyvault secret set --vault-name "$(${{ variables.podkv }})" --name "$name" --value "$value" --expires $expiryDate
     secretValue=$(az keyvault secret show --vault-name "$(${{ variables.podkv }})" --name "$name" --query value --output tsv)
     if [[ "$secretValue" != "$value" ]]; then
       echo "Failed to create secret $secretName in Key Vault $keyVaultName"
     exit 1
     fi
     if [[ "$secretValue" == "$value" ]]; then
       echo "Created secret $secretName in Key Vault $keyVaultName, hence cleaning the variable from the keyvault"
       az pipelines variable-group variable delete --group-id $groupID --name "$name --yes"
     fi
   fi
   done

Updated script,

where the secret is getting updated each time even though the same value is in the file.

while IFS= read -r line    
do    
  name=$(echo "$line" | cut -d '=' -f 1)    
  value=$(echo "$line" | cut -d '=' -f 2)    
  if ! az keyvault secret show --vault-name "$(${{ variables.podkv }})" --name "$name" &>/dev/null; then    
    expiryDate=$(date -u -d '+2 years' '+%Y-%m-%dT%H:%MZ')    
    az keyvault secret set --vault-name "$(${{ variables.podkv }})" --name "$name" --value "$value" --expires $expiryDate --output none  
    secretValue=$(az keyvault secret show --vault-name "$(${{ variables.podkv }})" --name "$name" --query value --output tsv)    
    if [[ "$secretValue" != "$value" ]]; then    
      echo "Failed to create secret $name in Key Vault ${{ variables.podkv }}"    
      exit 1    
    else    
      echo "Created secret $name in Key Vault ${{ variables.podkv }}, hence cleaning the variable from the keyvault"    
    fi    
  else    
    echo "The secretName $name already exists, so checking if the secret value is the same"    
    existing_secretvalue=$(az keyvault secret show --name "$name" --vault-name "$(${{ variables.podkv }})" --query "value")    
    if [[ "$existing_secretValue" != "$value" ]]; then    
      echo "there is secret value change for the secretname $name , so Creating a new secret value for the existing secret $name"    
      expiryDate=$(date -u -d '+2 years' '+%Y-%m-%dT%H:%MZ')    
      az keyvault secret set --vault-name "$(${{ variables.podkv }})" --name "$name" --value "$value" --expires $expiryDate --output none  
      new_secretValue=$(az keyvault secret show --vault-name "$(${{ variables.podkv }})" --name "$name" --query value --output tsv)    
      if [[ "$new_secretValue" != "$value" ]]; then    
        echo "Failed to create secret $name in Key Vault ${{ variables.podkv }}"    
        exit 1    
      else    
        echo "Created secret $name in Key Vault ${{ variables.podkv }}, hence cleaning the variable from the keyvault"    
      fi  
    fi  
  fi  
done < "$(keyvault.secureFilePath)"
Improve this question
1
  • Finally planned to use the below script to read values from library group and create the values in key vault - you should be doing the opposite: create variable groups with variables linked from an Azure Keyvault. Keep in mind that once you create a variable group with a secret variable you won't be able to retrieve it using the Azure DevOps portal. Commented Jul 22, 2024 at 8:44

2 Answers 2

Reset to default
1

when we are masking the secret values in Library group (by locking), the secrets are getting created with null value only.

This is an expected behavior. When we set the secret variable in Variable Group, we are not able to use Rest API or Azure DevOps CLI to get the variable. It will show as NULL Value.

In this case, I am afraid that the variable group method can not meet your requirement.

is there any way to retrieve the values secret variables

Based on your description, I suggest that you can save the variables in a file and add the file to Azure DevOps Secure file(Pipelines -> Library -> Secure file).

The secure file can only be downloaded with the Download secure file v1 task in Azure Pipelines.

Here are the steps:

Step1: We can create a file and save the variables.

For example:

test.txt

var1=value1
var2=value2
var3=value3

Step2: Upload the file to secure file.

For example:

enter image description here

Step3: We can use the Download secure file v1 task to download the secure file and then use the bash script to loop the values in the file. Finally we can set the Key vault value.

Here is an example:

steps:
- task: DownloadSecureFile@1
  name: keyvault
  inputs:
    secureFile: 'test.txt'
- bash: |
    while IFS= read -r line
    do
        name=$(echo "$line" | cut -d '=' -f 1)
        value=$(echo "$line" | cut -d '=' -f 2)

      if ! az keyvault secret show --vault-name "$(${{ variables.podkv }})" --name "$name" &>/dev/null; then
      expiryDate=$(date -u -d '+2 years' '+%Y-%m-%dT%H:%MZ')
      az keyvault secret set --vault-name "$(${{ variables.podkv }})" --name "$name" --value "$value" --expires $expiryDate
      secretValue=$(az keyvault secret show --vault-name "$(${{ variables.podkv }})" --name "$name" --query value --output tsv)
      if [[ "$secretValue" != "$value" ]]; then
        echo "Failed to create secret $secretName in Key Vault $keyVaultName"
      exit 1
      fi
      if [[ "$secretValue" == "$value" ]]; then
        echo "Created secret $secretName in Key Vault $keyVaultName, hence cleaning the variable from the keyvault"
        az pipelines variable-group variable delete --group-id $groupID --name "$name --yes"
      fi
    fi
    done < "$(keyvault.secureFilePath)"

In this case, the variable value will not be exposed during the pipeline run. And the secure file will be automatically deleted(On agent machine) after Pipeline run completing.

For more detailed info, you can refer to this doc: Use secure files

create them in key vault only if the secret value is different(if existing)

You can add the Azure CLI: Retrieve a secret from Key Vault to bash loop to compare secret value and current variable value.

Update:

I can reproduce the same issue when using the same bash script.

The cause of the issue is that when we use the Azure CLI to get the existing secret value. it will automatically add \r at the end of the value.

This line:

existing_secretvalue=$(az keyvault secret show --name "$name" --vault-name "$(${{ variables.podkv }})" --query "value")

For example:

enter image description here

To solve this issue, you can add --output tsv to format the output of the secret value.

For example:

az keyvault secret show --vault-name "$(${{ variables.podkv }})" --name "$name" --query value --output tsv

Update2:

it will not pick the value for variable which is having special character "=" in its value.

To solve this issue, you can use the following bash script to read the varaibles in file.

while IFS=$'=' read -r key values;
do    
  name=$key
  value=$values   
  echo $name
  echo $value
done < "$(keyvault.secureFilePath)"

Result:

enter image description here

The echo command is used to confirm if the value is correct. You can remove it in your bash script.

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

This Solution worked. But only challenge i am facing is, each time its creating new version of the secret even if the value in the securefile for that variable is same.
added my script to the updated script part of question
@Vowneee Thanks for your sharing. I will check the script further. If there is any update, I will let you know.
@Vowneee I can reproduce the same issue as yours. The cause of the issue is that the output of the Azure CLI: az keyvault secret show --name "$name" --vault-name "$(${{ variables.podkv }})" --query "value" will add additional \r at the end of the string. We need to format it. Please refer to the updated answer.
Nice.. I accepted the answer as well. One thing missing is its not cleaning up the securefile added from AzureDevops Securefile window, once the secrets are created. which may confuse our users whether they can upload new file there with same name or not (we have some condition to have the securefile naming, so keeping the securefile again will mislead the users second time when they tried to update the keyvaults.
|
-1

My requirement is to automate the creation of secrets in Keyvault without exposing the values of these secrets during the pipeline execution

When using Azure CLI you can use the --output none output format to keep sensitive information from being displayed in your console.

Example:

az keyvault secret set ... --output none

See None output format for more details.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.