AzureBlob Upload ERROR:The specified blob already exists

Question

I am trying to upload file to Azure container daily.

I got an Error:"The specified blob already exists" when uploading file with same file( I want to overwrite the file)

from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient

conn_str = yml['AZURE_BLOB']['CONN_STR']
container_name = yml['AZURE_BLOB']['CONTAINER_NAME']

# Create the BlobServiceClient that is used to call the Blob service for the storage account
blob_service_client = BlobServiceClient.from_connection_string(conn_str=conn_str)

# Create a blob client using the local file name as the name for the blob
blob_client = blob_service_client.get_blob_client(container=container_name, blob=destination_file_name)

# Upload the created file
data = fs.open(source_path,mode='rb').read()
blob_client.upload_blob(data)

print(destination_file_name+'\t......[DONE]')

Error message:

azure.core.exceptions.ResourceExistsError: The specified blob already exists.
RequestId:13d062cd-801e-00a4-77c7-a81c56000000
Time:2019-12-02T04:18:06.0826908Z
ErrorCode:BlobAlreadyExists
Error:None

Daggna

Have you checked that the file is already in blob storage? – Roderich

I checked, the file is exists. But I want to overwriter. – Daggna

See my answer below, and please let me know if any feedback:). – Roderich

Answer 1

If you want to overwrite the existing blob using Blob storage client library v12, just add overwrite=True in the upload_blob method.

Here is the sample code:

from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient

conn_str = "xxx"
container_name = "test6"

blob_service_client = BlobServiceClient.from_connection_string(conn_str=conn_str)
blob_client = blob_service_client.get_blob_client(container=container_name,blob="a1.txt")

with open("F:\\temp\\a1.txt","rb") as data:
    blob_client.upload_blob(data,overwrite=True)

print("**completed**")

After executing the code, the new blob is uploaded and the existing blob can be overwritten. Screenshot as below:

Roderich

Do you get the message like this "Tuple timeout setting is deprecated" – Daggna

@EEEEH, I didn't see this message. For which line? – Roderich

"blob_client.upload_blob(data, overwrite=True)" The message show up after uploading file success. Is it just a warning message yet? – Daggna

No matter why my promble sloved! tks – Daggna

Answer 2

Check out this blog post about a known issue.

This is a known issue with development storage. This happens when there are multiple threads launched to upload the blocks (which constitute the blob). Basically what is happening is that development storage makes use of SQL Server as the data store. Now first thing it does is makes an entry into the table which stores blob information. If there are multiple threads working then all of these threads will try to perform the same operation. After the first thread succeeds, the subsequent threads will result in this exception being raised.

Shira

its 2022 and we encountered this problem in Production , what should we do to solve this ? In my Visual Studio I dont have this problem but when we publish it to azure machine, the problem occurs – Jala

@Jala Might be related with encryption key/token to upload, try some variations with this method learn.microsoft.com/en-us/python/api/azure-storage-blob/… – Shira

thanks, it was only a bug on our side with folder naming – Jala