How to upload dynamic VHD to Azure using python and rest API calls?

Question

https://github.com/Microsoft/azure-vhd-utils is written in Go. Add-AzureRMVhd is the powershell cmd. Similarly, is there a python alternative that uploads dynamic VHD files and does checksum verification?

    #Working code to list blobs using GET API:
    import requests
    import datetime
    import hmac
    import hashlib
    import base64

    storage_account_name = 'abcd'
    storage_account_key = '4********************************************$'
    api_version = '2018-03-28'
    request_time = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')

    string_params = {
            'verb': 'GET',
            'Content-Encoding': '',
            'Content-Language': '',
            'Content-Length': '',
            'Content-MD5': '',
            'Content-Type': '',
            'Date': '',
            'If-Modified-Since': '',
            'If-Match': '',
            'If-None-Match': '',
            'If-Unmodified-Since': '',
            'Range': '',
            'CanonicalizedHeaders': 'x-ms-date:' + request_time + '\nx-ms-version:' + api_version + '\n',
        'CanonicalizedResource': '/' + storage_account_name + '/containername\ncomp:list\nrestype:container'
    }

    string_to_sign = (string_params['verb'] + '\n' 
                                        + string_params['Content-Encoding'] + '\n'
                                        + string_params['Content-Language'] + '\n'
                                        + string_params['Content-Length'] + '\n'
                                        + string_params['Content-MD5'] + '\n' 
                                        + string_params['Content-Type'] + '\n' 
                                        + string_params['Date'] + '\n' 
                                        + string_params['If-Modified-Since'] + '\n'
                                        + string_params['If-Match'] + '\n'
                                        + string_params['If-None-Match'] + '\n'
                                        + string_params['If-Unmodified-Since'] + '\n'
                                        + string_params['Range'] + '\n'
                                        + string_params['CanonicalizedHeaders']
                                        + string_params['CanonicalizedResource'])

    signed_string = base64.b64encode(hmac.new(base64.b64decode(storage_account_key), msg=string_to_sign.encode('utf-8'), digestmod=hashlib.sha256).digest()).decode()

    headers = {
            'x-ms-date' : request_time,
            'x-ms-version' : api_version,
            'Authorization' : ('SharedKey ' + storage_account_name + ':' + signed_string)
    }



    url = ('https://' + storage_account_name + '.blob.core.windows.net/containername?restype=container&comp=list')

    r = requests.get(url, headers = headers)

    print(r.content)

Is this the right canonicalized resource to upload a page blob? 'CanonicalizedResource': '/' + storage_account_name + '/containername/vhdname.vhd'

#Failing PUT request to upload page blob
import requests
import datetime
import hmac
import hashlib
import base64

storage_account_name = 'abc'
storage_account_key = '4*******************************='
api_version = '2018-03-28'
request_time = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')

string_params = {
        'verb': 'PUT',
        'Content-Encoding': '',
        'Content-Language': '',
        'Content-Length': '',
        'Content-MD5': '',
        'Content-Type': '',
        'Date': '',
        'If-Modified-Since': '',
        'If-Match': '',
        'If-None-Match': '',
        'If-Unmodified-Since': '',
        'Range': '',
        'CanonicalizedHeaders': 'x-ms-blob-type:PageBlob' + '\nx-ms-date:' + request_time + '\nx-ms-version:' + api_version + '\n',
        'CanonicalizedResource': '/' + storage_account_name + '/containername/vhdname.vhd'
}

string_to_sign = (string_params['verb'] + '\n' 
                                    + string_params['Content-Encoding'] + '\n'
                                    + string_params['Content-Language'] + '\n'
                                    + string_params['Content-Length'] + '\n'
                                    + string_params['Content-MD5'] + '\n' 
                                    + string_params['Content-Type'] + '\n' 
                                    + string_params['Date'] + '\n' 
                                    + string_params['If-Modified-Since'] + '\n'
                                    + string_params['If-Match'] + '\n'
                                    + string_params['If-None-Match'] + '\n'
                                    + string_params['If-Unmodified-Since'] + '\n'
                                    + string_params['Range'] + '\n'
                                    + string_params['CanonicalizedHeaders']
                                    + string_params['CanonicalizedResource'])

signed_string = base64.b64encode(hmac.new(base64.b64decode(storage_account_key), msg=string_to_sign.encode('utf-8'), digestmod=hashlib.sha256).digest()).decode()

headers = {
        'x-ms-date' : request_time,
        'x-ms-version' : api_version,
        'Content-Length' : '0',
        'x-ms-blob-type': 'PageBlob',
        'Authorization' : ('SharedKey ' + storage_account_name + ':' + signed_string)
}

url = ('https://' + storage_account_name + '.blob.core.windows.net/containername/vhdname.vhd')
r = requests.get(url, headers = headers)

print(r.content)

Tom Sun · Accepted Answer · 2018-09-25 09:03:07Z

0

is there a python alternative that uploads dynamic VHD files

We use Azure python sdk to upload the VHD file to Azure storage.

block_blob_service = BlockBlobService(account_name='accountname', account_key='accountkey') 
block_blob_service.create_blob_from_path(container_name, local_file_name, full_path_to_file)

For more information, please refer to the azure offical tutorial.

does checksum verification?

Yes, azure Blob service provides mechanisms to ensure data integrity both at the application and transport layers. This post will detail these mechanisms from the service and client perspective. MD5 checking is optional on both PUT and GET operations.

For more information, please refer to this blog.

answered Sep 25, 2018 at 9:03

Tom Sun

24.6k3 gold badges36 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

neo_phyte Over a year ago

Thanks for the response! Had tried block_blob_service.create_blob_from_path(container_name, local_file_name, full_path_to_file), however it fails to upload dynamic vhds ie wont expand the image correctly. Would be grateful for a sample "post" rest API on microsoft vhd page blob as it isn't functioning correctly.

Tom Sun Over a year ago

however it fails to upload dynamic vhds ie wont expand the image correctly Is there any error information? If az cli command is possible, you could use the

az storage blob upload --account-name mystorageaccount \     --account-key key1 \     --container-name mydisks \     --type page \     --file /path/to/disk/mydisk.vhd \     --name myDisk.vhd

We also could call the az command with python, for more information please refer to another SO thread

neo_phyte Over a year ago

Thanks for replying! As given in the documentation, Azure does not support uploading of dynamic vhd files through python sdk(allows only static)..hence I want to implement the same using rest calls(curl cmds) in my python script. Is it possible to do so without the Microsoft Azure storage emulator? Referring learn.microsoft.com/en-us/rest/api/storageservices/put-page

Tom Sun Over a year ago

Yes, the rest API could be used for Azue storage account.

neo_phyte Over a year ago

Any inputs on the API call to be used to upload vhd page blob(PUT request) will be much appreciated!

|

Collectives™ on Stack Overflow

How to upload dynamic VHD to Azure using python and rest API calls?

1 Answer 1

7 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related