How to get all files from a directory in Azure BLOB using ListBlobsSegmentedAsync

Question

While trying to access all files of the Azure blob folder, getting sample code for container.ListBlobs(); however it looks like an old one.

Old Code : container.ListBlobs();

New Code trying : container.ListBlobsSegmentedAsync(continuationToken);

I am trying to use the below code :

container.ListBlobsSegmentedAsync(continuationToken);

Folders are like :

Container/F1/file.json
Container/F1/F2/file.json
Container/F2/file.json

Looking for the updated version to get all files from an Azure folder. Any sample code would help, thanks!

Polanco

If the my answer is helpful, you can mark it as answer to help other community members find the helpful information quickly, thanks. – Ibnrushd

Answer 1

Update: Getting all files name from a directory with Azure.Storage.Blobs v12 - Package

var storageConnectionString = "DefaultEndpointsProtocol=...........=core.windows.net";
var blobServiceClient = new BlobServiceClient(storageConnectionString);

//get container
var container = blobServiceClient.GetBlobContainerClient("container_name");

List<string> blobNames = new List<string>();

//Enumerating the blobs may make multiple requests to the service while fetching all the values
//Blobs are ordered lexicographically by name
//if you want metadata set BlobTraits - BlobTraits.Metadata
var blobHierarchyItems = container.GetBlobsByHierarchyAsync(BlobTraits.None, BlobStates.None, "/");

await foreach (var blobHierarchyItem in blobHierarchyItems)
{
    //check if the blob is a virtual directory.
    if (blobHierarchyItem.IsPrefix)
    {
        // You can also access files under nested folders in this way,
        // of course you will need to create a function accordingly (you can do a recursive function)
        // var prefix = blobHierarchyItem.Name;
        // blobHierarchyItem.Name = "folderA\"
        // var blobHierarchyItems= container.GetBlobsByHierarchyAsync(BlobTraits.None, BlobStates.None, "/", prefix);     
    }
    else
    {
        blobNames.Add(blobHierarchyItem.Blob.Name);
    }
}

There are more option and example you can find it here.

This is the link to the nuget package.

Pricking

Answer 2

C# code:

   //connection string
    string storageAccount_connectionString = "**NOTE: CONNECTION STRING**";

    // Retrieve storage account from connection string.
    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(storageAccount_connectionString);

    // Create the blob client.
    CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

    // Retrieve reference to a previously created container.
    CloudBlobContainer container = blobClient.GetContainerReference("**NOTE:NAME OF CONTAINER**");
    //The specified container does not exist

    try
    {
        //root directory
        CloudBlobDirectory dira = container.GetDirectoryReference(string.Empty);
        //true for all sub directories else false 
        var rootDirFolders = dira.ListBlobsSegmentedAsync(true, BlobListingDetails.Metadata, null, null, null, null).Result;

        foreach (var blob in rootDirFolders.Results)
        {
             Console.WriteLine("Blob", blob);
        }

    }
    catch (Exception e)
    {
        //  Block of code to handle errors
        Console.WriteLine("Error", e);

    }

Clincher

Answer 3

Here is the code for the Answer :

private async Task<List<IListBlobItem>> ListBlobsAsync(CloudBlobContainer container)
{
    BlobContinuationToken continuationToken = null;
    List<IListBlobItem> results = new List<IListBlobItem>();
    do
    {
       bool useFlatBlobListing = true;
       BlobListingDetails blobListingDetails = BlobListingDetails.None;
       int maxBlobsPerRequest = 500;
       var response = await container.ListBlobsSegmentedAsync(BOAppSettings.ConfigServiceEnvironment, useFlatBlobListing, blobListingDetails, maxBlobsPerRequest, continuationToken, null, null);
            continuationToken = response.ContinuationToken;
            results.AddRange(response.Results);
        }
     while (continuationToken != null);
     return results;
}

And then you can return values like:

IEnumerable<IListBlobItem> listBlobs = await this.ListBlobsAsync(container);
foreach(CloudBlockBlob cloudBlockBlob in listBlobs)
  {
     BOBlobFilesViewModel boBlobFilesViewModel = new BOBlobFilesViewModel
     {
          CacheKey = cloudBlockBlob.Name,
          Name = cloudBlockBlob.Name
      };
      listBOBlobFilesViewModel.Add(boBlobFilesViewModel);
   }
//return listBOBlobFilesViewModel;

Polanco

Answer 4

The method CloudBlobClient.ListBlobsSegmentedAsync is used to return a result segment containing a collection of blob items in the container.

To list all blobs, we can use ListBlobs method,

Here is a demo for your reference:

    public static List<V> ListAllBlobs<T, V>(Expression<Func<T, V>> expression, string containerName,string prefix)
    {
        CloudStorageAccount storageAccount = CloudStorageAccount.Parse("YourConnectionString;");

        CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();

        CloudBlobContainer container = cloudBlobClient.GetContainerReference(containerName);
        container.CreateIfNotExists();

        var list = container.ListBlobs(prefix: prefix,useFlatBlobListing: true);

        List<V> data = list.OfType<T>().Select(expression.Compile()).ToList();
        return data;
    }

Usage and screenshots:

List all blobs' names under one folder:

List all blobs' URLs under one folder:

Ibnrushd

Thanks, but I resolved it using ListBlobsSegmentedAsync and my custom logic, however, ListBlobs is no more in use as far as I have tried.. :) – Polanco

ListBlobs is depreciated and removed from Azure SDK. – Mccomas