Download all blobs files locally from azure container using python

Question

I'm using python 3.6 in visual studio and I want to download all blobs files from my azure container in single folder. This is my code but the problem is, that it downloads 1 blob file in the folder and then when downloading the second file it overwrite the first file and in the end I only have the last blob in my local folder. How can I download all the blobs files at once in a single folder?

from azure.storage.blob import BlockBlobService
block_blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)

generator = block_blob_service.list_blobs(CONTAINER_NAME)
        for blob in generator:
            block_blob_service.get_blob_to_path(CONTAINER_NAME, blob.name, LOCAL_FILE)

Cuenca

Do you mean to download them and append the blobs into a single file? – Farthermost

You'd need to combine them yourself. Or maybe download them to a single stream. But the get_blob_to_path() method isn't going to append to files. – Marcin

Have you tried to pass open_mode='ab' to get_blob_to_path? – Farthermost

Answer 1

Based on my understanding, I think there are two solutions for your needs.

1. Download all blobs from a container, and write these blob content to a single file via the method get_blob_to_bytes or get_blob_to_stream, please see my sample code as below.

   from azure.storage.blob import BlockBlobService
   
   block_blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)
   
   generator = block_blob_service.list_blobs(CONTAINER_NAME)
   
   fp = open('<your-local-file-name>', 'ab')
   
   for blob in generator:
       # Using `get_blob_to_bytes`
       b = service.get_blob_to_bytes(container_name, blob.name)
       fp.write(b.content)
       # Or using `get_blob_to_stream`
       # service.get_blob_to_stream(container_name, blob.name, fp)
   
   fp.flush()
   fp.close()
   

2. Download all blobs from a container, and write these blobs into a zip file via the method get_blob_to_bytes, please see my sample code below.

   from azure.storage.blob import BlockBlobService
   import zipfile
   
   block_blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)
   
   generator = block_blob_service.list_blobs(CONTAINER_NAME)
   
   zf = zipfile.ZipFile(CONTAINER_NAME+'.zip', 
                mode='w',
                compression=zipfile.ZIP_DEFLATED, 
                )
   
   for blob in generator:
       b = service.get_blob_to_bytes(container_name, blob.name)
       zf.writestr(blob.name, b.content)
   
   zf.close()
   

Hope it helps. Any concern, please feel free to let me know.

Herdsman

Perfect answer..!! This is what I was looking. I think question should be edited so that google can index it. – Maxama

Answer 2

In the azure-storage-blob version 12.9.0 , I used the below script :

        from azure.storage.blob import BlobServiceClient
        import zipfile
        conn_str = ''
        block_blob_service = BlobServiceClient.from_connection_string(conn_str)
        generator = block_blob_service.get_container_client(container_name)
        my_blobs = generator.list_blobs()

        zf = zipfile.ZipFile("data"+'.zip', mode='w', compression=zipfile.ZIP_DEFLATED)

        for blob in my_blobs:
                bytes = generator.get_blob_client(blob.name).download_blob().readall()
                zf.writestr(blob.name, bytes)

        zf.close()

October