How can I open a writeable stream to a new Blob using the new Azure Storage SDK v12?

Question

I am using Azure Storage SDK v12 and I am looking for a way to open a stream to specific Blob, like in the previous versions:

CloudBlobClient cloudBlobClient = account.CreateCloudBlobClient();

CloudBlobContainer container = cloudBlobClient.GetContainerReference("zipfiles");

var blob = container.GetBlockBlobReference(Guid.NewGuid().ToString());

// Here is the the functionality I am looking for: OpenWrite that returns a Stream.
using (Stream blobStream = blob.OpenWrite())
{
  ....
}

The problem is that I do not find the OpenWrite method in the new API that would get me access to a Stream.

Question

How can I open a writeable stream to a Blob using the new Azure Storage SDK v12?

---

Workaround

Since I am working with Azure Functions, I have a workaround that involves using the Blob as Out parameter of my function:

public async Task RunAsync(
            [BlobTrigger("data/{listId}/{name}.txt", Connection = "storageAccountConnectionString")]
            Stream sourceStream,
            string listId,
            string name,
            [Blob("data/{listId}/processed/{name}.txt", FileAccess.Write, Connection = "storageAccountConnectionString")] 
            Stream destinationStream)
{
    //  Here I use the destinationStream instance to upload data as it is being processed. It prevents me from loading everything in memory before calling the UploadAsync method of a BlobClient instance.
}

Macdonald

Answer 1

You need to use GetBlockBlobClient extension method from the Azure.Storage.Blobs.Specialized namespace:

public async Task<Stream> CreateOutputStream(string fullFilePath)
{
    var blockBlobClient = _blobContainerClient.GetBlockBlobClient(fullFilePath);
    var stream = await blockBlobClient.OpenWriteAsync(true);

    return stream;
}

Rosalie

Does OpenWriteAsync() allow setting AccessTier like you can with UploadAsync()? I can't find it and I don't know if it's correct in terms of performance/billing to separately invoke SetAccessTierAsync(). – Insure

Answer 2

Try this

BlobContainerClient containerClient = await blobServiceClient.CreateBlobContainerAsync("zipfiles");
BlobClient blobClient = containerClient.GetBlobClient(Guid.NewGuid().ToString());

using(FileStream strem = File.OpenRead("10_million_lines.file")){
   await blobClient.UploadAsync(strem , true);
}

Outsail

This method uploads the content of a Stream to the Blob which means you have to have everything in your Stream ready, which is problematic when you have a lot data to write. I want to get a Stream from the Blob and use it to write data as I need. – Macdonald

@Kzrystof You dont have to have it ready in your stream. This is why its stream so it can read blocks and upload. Can you provide me more info about your original stream? Is it file? or what? – Outsail

I am extracting / filtering information from a file in a blob that has 10+ millions lines and want to upload it in different blobs. – Macdonald

@Kzrystof I go it. I will have a look – Outsail

@Kzrystof if you need just copy why dont you use BlobClient.StartCopyFromUri ? If its permission issue then you can create SharedAccessBlobPolicy to allow read it. – Outsail

@Kzrystof, I guess the openwrite method may be added in the next version because I see the openread method is added in the latest preview version 12.5.0-preview.6. – Raskin

@VolodymyrBilyachat For now, since I am using Azure Function, I was able to use an Out binding to a Blob which gives me an instance of Stream to the Blob as a parameter of my Run method. – Macdonald