How read all files from azure blob storage in C# Core

Question

I want to read files from an azure blob storage (the files inside the folder), the blob storage contains many folders. I want to read my folder 'blobstorage' ,it contains many JSON files performing .read to each file and some manipulations. I tried many code that did not work:

 CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
            CloudBlobContainer container = blobClient.GetContainerReference($"blobstorage");

The above code uses 'Microsoft.WindowsAzure.Storage' nuget package. This code is not working as expected. In many questions and answers found in stack overflow I found that most of them are outdated and does not work. Note: if any nuget mention that also bcs they are many packages

Consist

Answer 1

I found the solution in this post and worked perfectly for me. You just have to read it as a normal stream after the download.

BlobServiceClient blobServiceClient = new BlobServiceClient("connectionString");
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("containerName");
BlobClient blobClient = containerClient.GetBlobClient("blobName.csv");
if (await blobClient.ExistsAsync())
{
  var response = await blobClient.DownloadAsync();
  using (var streamReader= new StreamReader(response.Value.Content))
  {
    while (!streamReader.EndOfStream)
    {
      var line = await streamReader.ReadLineAsync();
      Console.WriteLine(line);
    }
  }
}

Mydriatic

Working, but Performance of this code is very poor. – Dibri

Answer 2

I don't see any option to list all blob using Microsoft.WindowsAzure.Storage package. If you can use Azure.Storage.Blobs package then try below code.

using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System;

namespace ConsoleApp2
{
    class Program
    {
        static string connectionString = "DefaultEndpointsProtocol=https;AccountName=storage******c9709;AccountKey=v**************************************;EndpointSuffix=core.windows.net";
        static string container = "azure-webjobs-hosts";
        static void Main(string[] args)
        {
            // Get a reference to a container named "sample-container" and then create it
            BlobContainerClient blobContainerClient = new BlobContainerClient(connectionString, container);
            blobContainerClient.CreateIfNotExists();
            Console.WriteLine("Listing blobs...");
            // List all blobs in the container
            var blobs = blobContainerClient.GetBlobs();
            foreach (BlobItem blobItem in blobs)
            {
                Console.WriteLine("\t" + blobItem.Name);
            }            
            Console.Read();
        }
    }
}

Output

You can also download the content of blob, Check this link

Impoverished

its just listing all files. i want read all files from specific folder – Consist

You can also download content of blob check updated answer, I added link – Impoverished

i don't all blob, only specified folder files and read filecontent, download is not possible – Consist

folder inside a container is just virtual folder, You can read all file and folder from particular container then you can filter and download only required folder JSON files. – Impoverished

Answer 3

If you have mass data to download and are looking for efficiency, you probably don't want to download them 1 by 1 on a single thread. Use multiple threads and async.

Here's a good read on the topic:

https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-scalable-app-download-files?tabs=dotnet

downloading 1000 files:

• with single-thread : 30seconds download time
• with multi-thread : 4seconds download time

Voluminous

Answer 4

You can find example code in the SDK github repo here for c#: https://github.com/Azure/azure-sdk-for-net/tree/Azure.Storage.Blobs_12.8.0/sdk/storage/Azure.Storage.Blobs/

You can use the following command to add the package to your dotNet Core project.

dotnet add package Azure.Storage.Blobs

Based on the examples there, you can enumerate the blobs and then read the one you're looking for.

Repetend

Answer 5

Try Below Code Out :

var connectionString = "your connection string";

        CloudStorageAccount storageacc = CloudStorageAccount.Parse(connectionString);
        //Create Reference to Azure Blob
        CloudBlobClient blobClient = storageacc.CreateCloudBlobClient();
        CloudBlobContainer container = blobClient.GetContainerReference("containerName");
        var blobs = container.GetDirectoryReference("FolderName").GetDirectoryReference("FolderName").ListBlobs().OfType<CloudBlockBlob>().ToList();
        Console.WriteLine("Total files found in directory: " + blobs.Count.ToString());
        var tempBlobList = blobs.Where(b => b.Name.Contains("fileName")).ToList();
        var response = await tempBlobList[0].DownloadTextAsync();

Amido