How to filter blobs in Azure SDK for Python

Question

I want to search for blobs in my Azure blob storage according to a specific tag (like: .name, .creation_date, .size...)

My current way is returning all blobs from the container with MyContainerClient.list_blobs and searching for the corresponding tag afterwards. Since my container stores around 800000 blobs, this takes me around 20 min, which is not usable for a live view of the content.

But I also found another ContainerClient function: .find_blobs_by_tags(filter_expression: str) where I can search for a specific blob whose tags matches the specified condition.

In the Azure API they specified this filter_expression as: ""yourtagname"='firsttag'" , therefore I specified: ""name"='example.jpg'" or ""creation_date"='2021-07-04 09:35:19+00:00'"

Azure SDK Python - ContainerClient.find_blobs_by_tag

Unfortunately I always get an error:

azure.core.exceptions.HttpResponseError: Error parsing query at or near character position 1: unexpected 'creation_time'
RequestId:63bd850b-401e-005f-745e-400d5a000000
Time:2022-03-25T15:40:22.4156367Z
ErrorCode:InvalidQueryParameterValue
queryparametername:where
queryparametervalue:'creation_time'='0529121f-7676-46c7-8a52-424664774240/0529121f-7676-46c7-8a52-424664774240.json'
reason:This query parameter value is invalid.
Content: <?xml version="1.0" encoding="utf-8"?>
<Error><Code>InvalidQueryParameterValue</Code><Message>Error parsing query at or near character position 1: unexpected &apos;creation_time&apos;
RequestId:63bd850b-401e-005f-745e-400d5a000000
Time:2022-03-25T15:40:22.4156367Z</Message><QueryParameterName>where</QueryParameterName><QueryParameterValue>&apos;creation_time&apos;=&apos;0529121f-7676-46c7-8a52-424664774240/0529121f-7676-46c7-8a52-424664774240.json&apos;</QueryParameterValue><Reason>This query parameter value is invalid.</Reason></Error>

Has someone experience with this Azure function calls?

Anupam Chand · Accepted Answer · 2022-03-25 17:41:58Z

0

Looking at the github code(in the find_blobs_by_tags function) , it says :

:param str filter_expression:
             The expression to find blobs whose tags matches the specified condition.
            eg. "\"yourtagname\"='firsttag' and \"yourtagname2\"='secondtag'"

Looks like you are missing the escape characters? Can you try including them in?

answered Mar 25, 2022 at 17:41

Anupam Chand

2,7691 gold badge8 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Alex Over a year ago

Hey, with this expression I'm not getting the InvalidQueryParameterValue error any more, unfortunately it is still not possible for me to get the blob paramters of the return iterator. When passing returned_iterator.next() I'm getting: This request is not authorized to perform this operation using this permission. Does somebody know how to change permission rights for the Storage Queue Data Contributor?

Anupam Chand Over a year ago

You will need atleast Storage Blob Data Reader role/permissions. You are trying to list blobs. Storage Queue Data Contributor is for storage queue only and has no access to the blobs. If you refer to learn.microsoft.com/en-us/azure/role-based-access-control/… it says that with Storage Blob Data Reader role you can read and list containers and blobs. Give that a go.

Collectives™ on Stack Overflow

How to filter blobs in Azure SDK for Python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related