How to create a blob in an Azure Storage Container using Python & Azure Functions

Question

I am having a lot of difficulty writing an API response as json to a blob within an Azure Storage Container. I have tried multiple solutions online but have not managed to see any through to success. I would like to share 2 attempts I have made and hopefully there is someone out there that can assist me in getting at least one methodology correct

Attempt/Method 1
I have tried to use a Service Principle to authenticate my BlobServiceClient from Azure-Storage-Blob. My service principal has been assigned the role of Storage Blob Data Contributor for the Container within which I am trying to create the blob. However on execution of the script I receive an error along the lines of "Unsupported Credential". Below is my script and the error:

My script and resulting error are:

import azure.functions as func
import requests
import json
import uuid
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
from msrestazure.azure_active_directory import ServicePrincipalCredentials
from azure.storage.common import TokenCredential

# Initialise parameters to obtain data from Rest API
url = "https://api.powerbi.com/v1.0/myorg/admin/groups?$top=1000&$expand=datasets,dataflows,reports,users,dashboards"
headers = {'Authorization': get_access_token()}

# Get response.  I want to save the response output to a blob.
response = requests.get(url, headers=headers)
response = response.json()

# Initialise parameters for credentials
CLIENT = "bxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx7" # Azure App/Service Principal ID
KEY = "Gxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1" # Azure App/Service Principal Key
TENANT_ID = "cxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx7" # Tenant where Storage Account is which is different to the Tenant the App resides
RESOURCE = f"https://storageaccountxxxxxxxxx.blob.core.windows.net"

# Create credentials & token
credentials = ServicePrincipalCredentials(
    client_id = CLIENT,
    secret = KEY,
    #tenant = TENANT_ID,
    resource = RESOURCE
)
tokenCre = TokenCredential(credentials.token["access_token"])

# Initialise parameters for BlobServiceClient
ACCOUNT_URL = "https://storageaccountxxxxxxxxx.blob.core.windows.net/pbiactivity" # includes container name at end of url

#Create BlobServiceClient
blobService = BlobServiceClient(account_url = ACCOUNT_URL, token_credential=tokenCre)

#Create blobClient
blobClient = BlobClient(account_url = RESOURCE,container_name=CONTAINER_NAME, blob_name="response.json", credential = tokenCre )    

#Upload response json as blob
blobClient.upload_blob(response, blob_type = "BlockBlob")

Click here for the error that comes after the upload_blob method call]1

Attempt/Method 2
In my second attempt I tried to create ,my BlobServiceClient using Azure-Storage-Blob using my storage account connection string. This method actually allows me to create containers, however when I try to upload a blob as in the script below, However I am unable to create blobs within a container as I get a 403 Forbidden response.

My script and resulting error are:

import requests
import json
import uuid
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient

# Initialise parameters to obtain data from Rest API
url = "https://api.powerbi.com/v1.0/myorg/admin/groups?$top=1000&$expand=datasets,dataflows,reports,users,dashboards"
headers = {'Authorization': get_access_token()}

# Get response.  I want to save the response output to a blob.
response = requests.get(url, headers=headers)
response = response.json()

# Initialise parameters
CONNECTION_STRING = "DefaultEndpointsProtocol=https;AccountName=storageaccountxxxxxxxxx;AccountKey=rxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxQ==;EndpointSuffix=core.windows.net"

# Create blobServiceClient from connection string
blobServiceClient = BlobServiceClient.from_connection_string(conn_str=CONNECTION_STRING)

#Create blobClient
blobClient = blobServiceClient.get_blob_client(container = "pbiactivity", blob = "response.json")

#Upload response json to blob
blobClient.upload_blob(response, blob_type = "BlockBlob")

Click Here for the errors that come after the upload_blob method call]2

SwethaKandikonda · Accepted Answer · 2022-02-09 02:05:21Z

1

Here is one of the workaround that worked for me:-

import os
import logging
from azure.storage.blob import BlobServiceClient, BlobClient

#Initialise parameters
url = "<YourURL>"
headers = {'Authorization': get_access_token()}

#Get response
response = requests.get(url, headers=headers)
response = response.json()

connectionString= "<Your_Connection_String>"
containerName = "<Name_of_your_container>"

blobServiceClient = BlobServiceClient.from_connection_string(connectionString)
blobContainerClient = blobServiceClient.get_container_client(containerName)

#To create Container (If the container has already been created you can ignore this)
#blobContainerClient.create_container()

#Create blobClient
blobClient = blobServiceClient.get_blob_client(container = "<Name_of_your_container>", blob = "response.json")

with open("response", "rb") as blob_file:
    blobClient.upload_blob(data=blob_file)

In my Storage Account:-

enter image description here

answered Feb 9, 2022 at 2:05

SwethaKandikonda

8,3002 gold badges8 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Nick Hall Over a year ago

Unfortunately when I use your code I am still getting an error as follows. It suggests the file response doesn't exist. To clarify I do not have a local file called response.json to open. I am trying to write my data direct from my response variable to the blob in json format. Have you any suggestions as to where I am going wrong? Result: Failure Exception: FileNotFoundError: [Errno 2] No such file or directory: 'response' Stack: File "/azure-functions-.......

SwethaKandikonda Over a year ago

Could you able to retrieve the value response.json to response?

Nick Hall Over a year ago

Yes I can get the value. But doesnt the statement open("response", "rb") require a file to open? I dont have or want to open a file. I just want to upload my response value to a blob

Collectives™ on Stack Overflow

How to create a blob in an Azure Storage Container using Python & Azure Functions

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related