How to delete a folder within an Azure blob container

Question

I have a blob container in Azure called pictures that has various folders within it (see snapshot below):

I'm trying to delete the folders titled users and uploads shown in the snapshot, but I keep the error: Failed to delete blob pictures/uploads/. Error: The specified blob does not exist. Could anyone shed light on how I can delete those two folders? I haven't been able to uncover anything meaningful via Googling this issue.

Note: ask me for more information in case you need it

Substage

Try to use this client azurestorageexplorer.codeplex.com – Assortment

Answer 1

Windows Azure Blob Storage does not have the concept of folders. The hierarchy is very simple: storage account > container > blob. In fact, removing a particular folder is removing all the blobs which start with the folder name. You can write the simple code as below to delete your folders:

CloudStorageAccount storageAccount = CloudStorageAccount.Parse("your storage account");
CloudBlobContainer container = storageAccount.CreateCloudBlobClient().GetContainerReference("pictures");
foreach (IListBlobItem blob in container.GetDirectoryReference("users").ListBlobs(true))
{
    if (blob.GetType() == typeof(CloudBlob) || blob.GetType().BaseType == typeof(CloudBlob))
    {
        ((CloudBlob)blob).DeleteIfExists();
    }
}

container.GetDirectoryReference("users").ListBlobs(true) lists the blobs start with "users" within the "picture" container, you can then delete them individually. To delete other folders, you just need to specify like this GetDirectoryReference("your folder name").

Titulary

That's true. Deleting the blobs inside the folder will delete the folder also. We need to just delete all blobs inside the folder. – Pilose

In the ForEach you can use CloudBlockBlob instead of IListBlobItem – Headrace

foreach (var item in items.OfType<CloudBlob>()) { item.Delete(); } – Humphreys

Will it also delete a folder say users2 ? – Bendy

For those looking for the latest (as of 2021) using Azure.Storage.Blobs v12, please check out the other answer by @Melih below: https://mcmap.net/q/300109/-how-to-delete-a-folder-within-an-azure-blob-container – George

THESE ANSWERS ARE NOT ACCURATE (as of March 2022) WHY can you CREATE a "folder" in an Azure Storage Blob Container (using either python or Azure Storage Explorer", and the folder PERSISTS even when EMPTY? WHY can I POPULATE this "/folder001/" then MOVE these files to ANOTHER "/folder002/", and I clearly see that the now-EMPTY /folder001/ CONTINUES TO PERSIST until I MANUALLY DELETE it? What are the SMEs NOT understanding about "folders" in Azure Storage Blob Containers?? – Punch

This is true if you have Hierarchical namespace disabled. If enabled, the folder will NOT be deleted if you delete the (only blob) in the folder. – Specific

@tom, can i know where to run that script? – Gisarme

@Gisarme sorry, I am not the author, just cleaned the answer up. – Carlstrom

you are wrong i have azure storage account , container , directory and then blob we can structure as per requirement – Tun

Answer 2

There is also a desktop storage explorer from Microsoft. It has a feature where you can select the virtual folder and then delete it effectively deleting all sub blobs.

https://azure.microsoft.com/en-us/features/storage-explorer/

Span

This does save from clicks/scripting, but does use the same enumerate-then-delete-one-by-one mechanics inside. So it may bomb a lot of delete operations and is sloooow,. – Herbage

May be slow, but has a "Copy AzCopy Command to Clipboard" feature that showed me how to do this on the command line :) – Sather

...added an answer to show how to do this with azcopy directly (which is fast). – Sather

Answer 3

In the latest repo, Azure.Storage.Blobs, it's very straightforward

var connectionString = "blob-connection-string";
var containerName = "container-name";
var folderPath = "folder1/subfolder/sub-subfolder";

var blobServiceClient = new BlobServiceClient(connectionString);
var blobContainerClient = blobServiceClient.GetBlobContainerClient(containerName);
var blobItems = blobContainerClient.GetBlobsAsync(prefix: folderPath);
await foreach (BlobItem blobItem in blobItems)
{
     BlobClient blobClient = blobContainerClient.GetBlobClient(blobItem.Name);
     await blobClient.DeleteIfExistsAsync();
}

As every blob has its own uri value, you can set prefix before querying so that it can fetch and delete blobs with the particular uri. The folder will disappear as the blobs get deleted.

Intrusion

can I know how to execute that script? – Gisarme

Unless: you cannot delete a non-empty folder, and cannot delete an empty folder too. I love these error messages. – Teferi

Answer 4

Let's start with an example how to delete a "folder" using ListBlobsSegmentedAsyc:

var container = // get container reference
var ctoken = new BlobContinuationToken();
do
{
    var result = await container.ListBlobsSegmentedAsync("myfolder", true, BlobListingDetails.None, null, ctoken, null, null);
    ctoken = result.ContinuationToken;
    await Task.WhenAll(result.Results
        .Select(item => (item as CloudBlob)?.DeleteIfExistsAsync())
        .Where(task => task != null)
    );
} while (ctoken != null);

What it does...

var ctoken = new BlobContinuationToken();

A "folder" may contain a lot of files. ListBlobSegmentedAsyc may return only a part of them. This token will store the info where to continue in next call.

var result = await container.ListBlobsSegmentedAsync("myfolder", true, BlobListingDetails.None, null, ctoken, null, null);

• First argument is the required blob name ("path") prefix.
• Second argument "useFlatBlobListing=true" tells the client to return all items in all sub folders. If set to false, it will run in "virtual folders" mode and behave like a file system.
• The ctoken will tell azure where to continue

For all arguments see https://learn.microsoft.com/en-us/dotnet/api/microsoft.windowsazure.storage.blob.cloudblobclient.listblobssegmentedasync?view=azure-dotnet for details.

(item as CloudBlob)?.DeleteIfExistsAsync()

Now we have a list of IListBlobItem in result.Results. Because an IListBlobItem is not guaranteed to be a deletable CloudBlob (e.g. it could be a virtual folder if we would have set useFlatBlobListing=false), we try to cast it and delete it if possible.

result.Results.Select(item => (item as CloudBlob)?.DeleteIfExistsAsync())

Triggers delete for all results and returns a list of tasks.

.Where(task => task != null)

If Results contained items we could not cast to CloudBlob, our list of tasks contains null values. We have to remove them.

... then we wait until all delete for current segment finished and continue with next segment if available.

Epiboly

Answer 5

Its because the "folders" don't actually exist. In an Azure storage account, you have containers which are filled with blobs. What you see visualized by clients as "folders" are the file names of the blobs in the account "pictures/uploads/". If you want to remove the "folder", you actually have to remove each of the blobs that are named with the same "path".

The most common approach is to get a list of these blobs then feed that to the delete blob call.

Feasible

Okay, but does the delete blob call scale to, say, 20M objects? – Substage

"scale" is a relative term. Can you delete all 20M blobs? yes, Can do you it in 1 sec? no. you'll be subject to the limits of the Storage API throttling. – Feasible

It wouldn't hurt to put a checkbox "select all" though – Mariellemariellen

Answer 6

The WindowsAzure.Storage package was split into separate packages at version 9.4.0. This means that the APIs used in the accepted answer have changed in the more recent Azure.Storage.Blobs package.

The approach below uses the APIs from the newer Azure.Storage.Blobs package, but still uses the same approach of accepted answer by listing all the blobs and then deleting them one at a time.

string ConnectionString = "<your connection string>";
string ContainerName = "<your container name>";

private BlobContainerClient ContainerClient()
{
    var client = new BlobContainerClient(ConnectionString, ContainerName);
    client.CreateIfNotExists();
    return client;
}

public async Task<List<BlobItem>> ListBlobsAsync(string folder)
{
    var c = ContainerClient();
    var enumerator = c.GetBlobsByHierarchyAsync(prefix: folder).GetAsyncEnumerator();

    var result = new List<BlobItem>();
    while (await enumerator.MoveNextAsync())
    {
        if (enumerator.Current.IsBlob)
            result.Add(enumerator.Current.Blob);
    }
    return result;
}

public async Task DeleteByFolderAsync(string folder)
{
    var c = ContainerClient();
    foreach (var blob in await ListBlobsAsync(folder))
    {
        await c.GetBlobClient(blob.Name).DeleteIfExistsAsync(DeleteSnapshotsOption.IncludeSnapshots);
    }
}

Rossiya

perfect! just what I was looking for – Liebig

Answer 7

Try using the Azure CLI

For example if you want delete a path starting with pictures/users here you can find all the blobs

export CONN_STRING="<YOUR-CONNECTION-STRING>"

az storage blob list -c mycontainer \
   --connection-string $CONN_STRING \
   --output tsv \
   --prefix pictures/users

Or maybe you want go straight and delete all of them:

az storage blob delete-batch -s mycontainer \
   --connection-string $CONN_STRING \
   --pattern pictures/users/*

Retorsion

Answer 8

You can also do it in Azure cloud shell; here is the command:

az storage blob delete-batch --source <blob-container> --account-name <blob-account> --pattern <folder-name>*

Ravi

Answer 9

Some simple code to achieve the desired behaviour:

    public static async Task DeleteFolder(string containerName, string folder)
    {
        CloudBlobContainer container = await GetContainerAsync(containerName);

        BlobResultSegment blobList = null;
        bool folderIsEmpty = false;

        while (!folderIsEmpty)
        {
            blobList = await container.ListBlobsSegmentedAsync(
                prefix: folder,
                useFlatBlobListing: true,
                blobListingDetails: BlobListingDetails.None,
                maxResults: null,
                currentToken: null,
                options: null,
                operationContext: null
            );

            folderIsEmpty = true;

            foreach (IListBlobItem item in blobList.Results)
            {
                folderIsEmpty = false;
                await ((CloudBlockBlob)item).DeleteIfExistsAsync();
            }
        }
    }

Quaternion

This is the answer I am looking for – Hew

Answer 10

with Azure CLI, the command with delete-batchshowing before delete recursively all blobs in a directory, but not delete the directory.

As it said before, the hierarchy is storage account > container > blob. So, you can treat a directory as a blob and use azcopycommand :

az storage azcopy blob delete --account-name MyAccount --container MyContainer --target MyDirectory 

Philomenaphiloo

Answer 11

I will add another way using azcopy on the command line. This is basically what the Microsoft Azure Storage Explorer desktop application does when you use it to delete files from a blob storage.

Assume you would like to delete (all files from) the subfolder l1:

azcopy remove \
  https://nameofyourstorageaccount.blob.core.windows.net/datasets/your/path/here/l1/ \
  --from-to=BlobTrash --recursive --log-level=INFO --dry-run

Check the output and then remove --dry-run to actually perform the deletion.

(In my case, this took about 8 seconds to delete around 20k files, so it is rather quick.)

Sather

Answer 12

If it's a one-time drag, simply install storage-explorer - it supports deleting a directory.

Leprose

Answer 13

In linux, one has to make use of quotes if they want to use a wildcard path i.e. '*'

az storage blob delete-batch -s mycontainer \
   --connection-string $CONN_STRING \
   --pattern 'pictures/users/*'

Akel

Just need a connection string, which can be obtained from Microsoft Entra ID. Works perfectly. – Motivity

Answer 14

Now you can use lifecycle management to delete file with prefixMatch and action to delete with property daysAfterModificationGreaterThan. Leave the rule active for about 24 hours. And it will do the job. Documentation of lifecycle management available at https://learn.microsoft.com/en-us/azure/storage/blobs/storage-lifecycle-management-concepts

Stpeter

Answer 15

Assuming you don't have thousands of records, and that you don't want to mess with scripts ... You can hit space bar then down arrow, then repeat, to select a bunch of files quicker than doing it with a mouse. This saved me some time anyway.

Saros

Answer 16

You can now use the delete activity in ADF to delete any file /blob. It will delete all the files inside

Jointly