Invalid client secret - Velero

11/3/2021

Using velero (v1.7) backup tool, I am trying to backup the cluster data and volume data as I installed velero with restic. I have deployed velero in a k8s cluster in Azure and I use Azure Blob container as backup storage location.

The credentials-velero file which I used to install velero server looks similar to:

AZURE_SUBSCRIPTION_ID=<subs-id>
AZURE_TENANT_ID=<tenant-id>
AZURE_CLIENT_ID=<client-id>
AZURE_CLIENT_SECRET=<client-secret>
AZURE_RESOURCE_GROUP=<rg>
AZURE_CLOUD_NAME=AzurePublicCloud

I used "az ad sp create-for-rbac --name "<service-principal-name>" --role "Contributor" --query 'password' -o tsv --scopes /subscriptions/$AZURE_SUBSCRIPTION_ID" command to get client secret and az ad sp list --display-name "<service-principal-name>" --query '[0].appId' -o tsv to get client ID.

I have created a scheduled backup which takes the full cluster backup once in a week. It's been 3 months that I installed velero and enabled the backup. But in recent days, I'm getting the following logs in velero pod:

level=error msg="Error getting backup store for this location" backupLocation=default controller=backup-sync error="rpc error: code = Unknown desc = azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/<subs-id>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<storage-acount-name>/listKeys?%24expand=kerb&api-version=2019-06-01: StatusCode=401 -- Original Error: adal: Refresh request failed. Status Code = '401'. Response body: {\"error\":\"invalid_client\",\"error_description\":\"AADSTSxyzabc: Invalid client secret is provided.\\r\\nTrace ID: <trace-id>\\r\\nCorrelation ID: <correlation-id>\\r\\nTimestamp: 2021-11-03 08:09:17Z\",\"error_codes\":[err-code],\"timestamp\":\"2021-11-03 08:09:17Z\",\"trace_id\":\"<trace-id>\",\"correlation_id\":\"<correlation-id>\",\"error_uri\":\"https://login.microsoftonline.com/error?code=<err-code>\"}" error.file="/go/src/velero-plugin-for-microsoft-azure/velero-plugin-for-microsoft-azure/object_store.go:217" error.function=main.getStorageAccountKey logSource="pkg/controller/backup_sync_controller.go:175"

In the azure portal, I have set the expiration of the client secret to about 2 years. But the velero logs shows that the invalid client secret has been provided.

So, can anyone please guide me to resolve this issue? Thanks in Advance.

-- Soundarya
azure-aks
kubernetes
velero

0 Answers