Create blob container in azure storage if it does not exists

Question

I am trying to create blob container in azure storage using python. I am using documentation provided by MSDN to integrate azure blob storage in my python program.

Here is the code:

connectStr = <connString>
blobServiceClient = BlobServiceClient.from_connection_string(connectStr)
containerName = "quickstart-azureStorage"
localFileName = "quickstart-1.txt"
blobClient = blobServiceClient.create_container(containerName)

create_container() is creating blob container first time, but it is giving me error second time.

I want to create blob container if it does not exists. If it exist then use existing blob container

I am using azure storage library version 12.0.0. i.e azure-storage-blob==12.0.0

I know we can do it for blob present in that container using below code, but I did not find anything for creating container itself.

Check blob exists or not:

blobClient = blobServiceClient.get_blob_client(container=containerName, blob=localFileName)
if blobClient:
    print("blob already exists")
else:
     print("blob not exists")

Exception:

RequestId:<requestId>
Time:2019-12-04T06:59:03.1459600Z
ErrorCode:ContainerAlreadyExists
Error:None

Sophiasophie

Answer 1

If you are working with a version of azure-storage-blob before 12.8, then a possible solution is to use the get_container_properties function, which will error if the container does not exist.

This solution was tested with version 12.0.0.

from azure.storage.blob import ContainerClient

container = ContainerClient.from_connection_string(connectStr, 'foo')

try:
    container_properties = container.get_container_properties()
    # Container foo exists. You can now use it.

except Exception as e:
    # Container foo does not exist. You can now create it.
    container.create_container()

---

If you are working with a version of azure-storage-blob after 12.8, then you can simply use the exist function, which will return true if the container exist, and false if the container doesn't exist.

This solution was tested with version 12.8.1.

from azure.storage.blob import ContainerClient

container = ContainerClient.from_connection_string(connectStr, 'foo')

if container.exists():
    # Container foo exists. You can now use it.

else:
    # Container foo does not exist. You can now create it.
    container.create_container()

Capacity

Answer 2

If you look at the documentation for create_container, it states that:

Creates a new container under the specified account. If the container with the same name already exists, the operation fails.

One possible solution to overcome this is to create the container and catch the error. If the container already exists, then you will get a Conflict (409) error code. Based on this you can determine if the container exists or not.

If downgrading the SDK is an option, you can use version 2.1 of Python Storage SDK. There the default behavior is not to throw an exception if the container exists. You can see the code for create_container here: https://github.com/Azure/azure-storage-python/blob/master/azure-storage-blob/azure/storage/blob/baseblobservice.py.

Eck

Thanks for your answer Gaurav. I am looking for function like CreateIfNotExists() (Available in C#) learn.microsoft.com/en-us/dotnet/api/… – Sophiasophie

Suggestions given by you is alternate way to achieve this. If python library does not contain any function like CrateIfNotExists(), then I will try your solution – Sophiasophie

I was also surprised that the Python SDK doesn't have an equivalent of CreateIfNotExists() method. May be submit an issue on the Github repo? – Eck

Answer 3

I you could accept the low version storage sdk, you could try the azure-storage-blob==2.1.0, there is a exists method to check if the blob or container exists. If exists it will return true or return false.

After the exists method create the container if it returns false, if return true just use the container.

Kennard

is this exist() method available in azure-storage-blob==12.0.0. I am using version 12.0.0 of azure-storage-blob – Sophiasophie

No this method is removed in the latest sdk. If you want to use the latest sdk, you have to use the Gaurav Mantri way to catch the exception code. – Kennard

@Prasad Telkikar,If these ways you don't want to use maybe you could use list_containers, then do the judgement in a for loop. – Kennard

Today I will try list_containers way and will check it in for loop. – Sophiasophie

You should know for now the latest sdk doesn't support this method for python, then you just need to decide if you could use the old sdk. If not , just try the other ways I said and choose a more acceptable approach. – Kennard

Answer 4

We no longer need to downgrade sdk for this. In azure-storage-blob, 12.8.* release, we now have an exists method on the ContainerClient object which could be used. [GitHub PR] for reference.

from azure.storage.blob import ContainerClient

# if you prefer async, use below import and prefix async/await as approp. 
# Code remains as is.
# from azure.storage.blob.aio import ContainerClient


def CreateBlobAndContainerIfNeeded(connection_string: str, container_name: str, 
                                   blob_name: str, blob_data):

    container_client = ContainerClient.from_connection_string(
        connection_string, container_name)

    if not container_client.exists():
        container_client.create_container()

    container_client.upload_blob(blob_name, blob_data)


if __name__ == "__main__":
    connection_string = "DefaultEndpointsProtocol=xxxxxxxxxxxxxxx"
    
    CreateBlobAndContainerIfNeeded(
        connection_string,
        "my_container",
        "my_blob_name.dat",
        "Hello Blob",
    )

Lopeared

Answer 5

Similar to @Paolo's great answer, you could wrap your code in a try..catch block, and catch the azure.core.exceptions.ResourceExistsError exception if the container resource already exists.

The difference here is that I am catching the specific exception raised, and @Paolo's answer is catching Exception, which is the base class for all exceptions. I find it clearer to catch the specific exception in this case, since its more distinct with what error I'm trying to handle.

Additionally, If the container doesn't exist, then azure.storage.blob.BlobServiceClient.create_container will create the container resource.

from azure.core.exceptions import ResourceExistsError

blob_service_client = BlobServiceClient.from_connection_string(connection_string)

try:
    # Attempt to create container
    blob_service_client.create_container(container_name)

# Catch exception and print error
except ResourceExistsError as error:
    print(error)

# Do something with container

Which will print this error from the exception:

The specified container already exists.
RequestId:5b70c1d8-701e-0028-3397-3c77d8000000
Time:2020-06-07T06:46:15.1526773Z
ErrorCode:ContainerAlreadyExists
Error:Non

We could also just ignore the exception completely with the pass statement:

try:
    # Attempt to create container
    blob_service_client.create_container(container_name)

# Catch exception and ignore it completely
except ResourceExistsError:
    pass

Oxazine

Answer 6

Modifying Paola's answer and Azure Storage Python SDK tutorial

connect_str = os.getenv('AZURE_STORAGE_CONNECTION_STRING')

# Create the BlobServiceClient object which will be used to create a container client
blob_service_client = BlobServiceClient.from_connection_string(connect_str)

# Create a unique name for the container
container_name = "foo"

# Create the container
try:
    container_client = blob_service_client.get_container_client(container_name)
    # Container foo exists. You can now use it.
except Exception as e:
    # Container foo does not exist. You can now create it.
    print(e)
    container_client = blob_service_client.create_container(container_name)

Module version

Name: azure-storage-blob
Version: 12.5.0

References:

Azure Storage Python SDK

Sample code

Michey