Can I upload a stream to Azure blob storage without specifying its length upfront?

Question

I don't know if it's relevant, but I am using Java with the azure-storage-android-0.2.0.aar for the upload.

I can upload files to Microsoft Azure blob storage

CloudBlockBlob blob = container.getBlockBlobReference("filename.ext");
blob.upload(inputStream, n);

where n is the length of the inputStream when it is derived from the file.

Here's my problem: I would like to stream directly, for example from the camera, which apparently isn't possible as Azure requires the length parameter for the upload, which is unknown while still streaming.

Is there a reason why I need to specify the length? (MD5?) And is there a way to upload while the stream is still being produced (which obviously is the idea of an InputStream in Java, the reason why InputStream does not have a length property)?

Vaas

Answer 1

We’ll log feature request to enable uploading from a stream without specifying length. For now, you may want to use the openOutputStream method which has a write method taking a byte[] or an int. An example using the int method is below:

    CloudBlockBlob blockBlob = container.getBlockBlobReference(‘myblob’); // assuming container was already created

    BlobOutputStream blobOutputStream = blockBlob.openOutputStream();
    ByteArrayInputStream inputStream = new ByteArrayInputStream(buffer); // assuming buffer is a byte[] with your data

    int next = inputStream.read();
    while (next != -1) {
          blobOutputStream.write(next);
          next = inputStream.read();
    }

    blobOutputStream.close();

Cuttle

@trailmax Hopefully that makes your aforementioned C# code easier. :) If you have any other issues like what you mentioned before, please do post an issue/feature request on our GitHub page or here on Stack Overflow. We monitor both regularly and are happy to answer questions and take feedback. – Cuttle

@EmilyGerner-Microsoft : I was curious whether Blob storage works the same way the service client work. DO I need to add a seperate async task to get the work done. – Twenty

@TerrilThomas Could you please clarify? Which storage library are you using (language and version)? Do you have a code snippet? Perhaps open a new question with this information on Stack Overflow or the appropriate GitHub repository so we can better help you. – Cuttle

@EmilyGerner-Microsoft I have used your demo to upload image. its working fine. But now, after uploading image i need its url to display, how i can do that? I mean how i can get URL of the uploaded image? – Tunnage

@iDroidExplorer This should be really simple. CloudBlockBlob has a getUri method. Is this what you need? – Cuttle

Hey, just wanted to comment it would still nice to be able to upload without needing the length to provide parity between the AWS sdk and our own. Is it as simple as just adding a new method, upload that takes a single input of a stream, where the function is the code you have provided here? – Haemophiliac

Not at all or we'd have done it. :) The length determines how we have to upload. Smaller blobs are done in a single put whereas larger blobs require chunking. If we don't know length we can't make an appropriate decision. – Cuttle

So the method your giving here should only be used if you know your file is small? – Haemophiliac

What do you mean? The library does chunking for big blobs so using any of these methods works for all sizes. – Cuttle

Answer 2

I did it using BlobAsyncClient below is the code for that:

BlobServiceAsyncClient blobServiceAsyncClient = blobServiceClientBuilder.buildAsyncClient();

BlobContainerAsyncClient blobContainerAsyncClient =
    blobServiceAsyncClient.getBlobContainerAsyncClient("container name");

BlobAsyncClient blobAsyncClient =
    blobContainerAsyncClient.getBlobAsyncClient("blob name");

ParallelTransferOptions parallelTransferOptions = new ParallelTransferOptions()
    .setBlockSizeLong(blockSize).setMaxConcurrency(5)
    .setProgressReceiver(bytesTransferred -> log.info("Uploaded bytes:{}", bytesTransferred));

Flux<ByteBuffer> data = Flux.just(ByteBuffer.wrap(byteArray));

blobAsyncClient.upload(data, parallelTransferOptions, true)
    .doOnError(throwable -> log.error("Error occurred while uploading:{}",
        throwable.getMessage()))
    .doOnSuccess(success -> {
      log.info("Blob is uploaded successfully!!:{}", blobAsyncClient.getBlobUrl());
    })
    .subscribe();

Agouti