Azure Table Storage Exception: 409 Conflict unexpected?

Question

I'm using WindowsAzure.Storage nuget package version 9.0.0.

Install-Package WindowsAzure.Storage -Version 9.0.0

The following code (table.CreateIfNotExistsAsync()) throws the error: The remote server returned an error: (409) Conflict.

CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference(tableName);
try
{
    if (await table.CreateIfNotExistsAsync())
    {
        log.Info(string.Format("Created Table named: {0}", tableName));
    }
}
catch (StorageException)
{
    log.Error("If you are running with the default configuration please make sure you have started the storage emulator. Press the Windows key and type Azure Storage to select and run it from the list of applications - then restart the sample.");
    throw;
}

return table;

If I check the StorageException details, I find this message: The table specified already exists.

Stack Trace:

   at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndExecuteAsync[T](IAsyncResult result) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line 57

This code works fine:

CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference(tableName);

try
{
    //if (await table.CreateIfNotExistsAsync())
    //{
    //    log.Info(string.Format("Created Table named: {0}", tableName));
    //}

    if (!table.Exists())
    {
        await table.CreateAsync();

        log.Info(string.Format("Created Table named: {0}", tableName));
    }
}
catch (StorageException)
{
    log.Error("If you are running with the default configuration please make sure you have started the storage emulator. Press the Windows key and type Azure Storage to select and run it from the list of applications - then restart the sample.");
    throw;
}

return table;

I know I have tables that already exists and they are NOT currently being deleted. Why am I getting this error? Since the table does exist, I would expect this to perform and existence check and just return true, not throw a storage exception.

Edit: This is how I'm creating the CloudStorageAccount

public static CloudStorageAccount CreateStorageAccountFromConnectionString(string storageConnectionString)
{
    CloudStorageAccount storageAccount;
    try
    {
        storageAccount = CloudStorageAccount.Parse(storageConnectionString);
    }
    catch (FormatException fe)
    {
        log.Error("Invalid storage account information provided. Please confirm the AccountName and AccountKey are valid in the app.config file - then restart the application.", fe);
        throw;
    }
    catch (ArgumentException ae)
    {
        log.Error("Invalid storage account information provided. Please confirm the AccountName and AccountKey are valid in the app.config file - then restart the sample.", ae);
        throw;
    }

    return storageAccount;
}

Storage connection string looks like this:

<add key="StorageConnectionString" value="DefaultEndpointsProtocol=https;AccountName=something;AccountKey=somekeygoeshere==" />

Veron

Answer 1

CreateIfNotExists is throwing the exception.
There was a change in implementation of the method CloudBlobContainer.CreateIfNotExists or CloudTable.CreateIfNotExists.

The implementation in the 7.1.2 storage client library is the following:

1. Make a call to check to see if the storage container or table exists (HTTP HEAD request).

2. If it exists do nothing.

3. If it doesn’t exist then make a call to create the container or table.
    

In the storage client library 8.1.1 the implementation is the following:
 

1. Make a call to create the container or table (HTTP PUT Request).

2. If an error was returned (HTTP 409) because the container or table already exists then do nothing. The error is handled.

3. If the container or table did not exist then create would have succeeded.

I checked with 9.0 and that behavior still exists

Ebarta

In 8.1.1, you say "The error is handled". By whom? The library? CreateIfNotExists should not throw a storage exception for me to handle if it does exist. It should just return false. – Veron

The application should not. It should be handled by the library. You will still see it in Fiddler. Have you checked the Fiddler? By the way SAS is method in which you are accessing Storage: learn.microsoft.com/en-us/azure/storage/common/… but if you are using the key then you have list permission – Ebarta

Why do we need CreateIfNotExists in this scenario if it throws!? What does it do beyond Create? – Hochheimer

The problem was the connectionstring for key1. I created a SAS with all permissions, and now it works as expected. But the conflict exception is just wrong! It should have said something about missing permissions. – Hochheimer

Plot twist: error or not, ApplicationInsights logs it as a failed request and you get pretty red logs! – Dorri

This is nuts. Why then is it called CreateIfNotExists? – Fuss

Very annoying. This leads to very high dependency failure rates in Application Insights. – Aeroneurosis

If you want to avoid the ugly red logs in application insights you can write your own CreateIfNotExists: if (!table.Exists()) { await table.CreateAsync(); } – Cam

@ShaneRaboin - Do you happen to know if this same behavior also exhibited by CloudFileDirectory.CreateIfNotExists() and/or CloudQueue.CreateIfNotExists() ? Your answer seems to indicate it may only be in those method implementations in CloudBlobContainer and CloudTable. – Undertow

Answer 2

A nice workaround could be:

if (!container.Exists())
        {
            container.CreateIfNotExists();
        }

Nudibranch

I think you can just safely call CreateAsync if you're wrapping it in a check. if (!table.Exists()) { await table.CreateAsync(); } – Cam

@CrhistianRamirez I think container.CreateIfNotExists(); is safer since (even though the chance might be small) if the table is created after you checked if it exists or not then you will get an exception in the CreateAsync which you have to handle yourself. – Grappa

Answer 3

How did you create CloudStorageAccount object? I think it was probably created via SAS which doesn't have List Tables permission, so table.Exists() always returns false per storage REST API design in order not to leak such information to unauthorized users. The reason why table.CreateIfNotExistsAsync() didn't meet this issue is that the method directly calls Create Table REST API without a prior List Tables checking, and 409 Conflict exception is swallowed inside table.CreateIfNotExistsAsync() method on purpose (in order to implement "create if not exists" functionality, since 409 Conflict exception means the table already exists so the method can just ignore the error and do nothing).

If my theory above applies to your case, please use a SAS with List Tables permission instead of the existing SAS, or use shared account key directly to create CloudStorageAccount object.

Abixah

What is SAS? I've added code for how I'm creating CloudStorageAccount. – Veron

table.Exists() returns true, as it should. – Veron

I created a SAS with all permissions. Worked on first attempt, then throwed. – Hochheimer

SAS is Shared Access Signature. Per my update code, you're using connection string rather than SAS, it's weird since I can't reproduce the issue from my side. – Abixah

@JasonEades I know it's been a while but you can filter out the 409s. See the following link: #45227939 & John Gardner's reply. – Ebarta