How can I get references to BlockBlob objects from CloudBlobDirectory.ListBlobs?

Question

I am using the Microsoft Azure .NET client libraries to interact with Azure cloud storage. I need to be able to access additional information about each blob in its metadata collection. I am currently using CloudBlobDirectory.ListBlobs() method to get a list of blobs in a particular directory of a directory structure I've devised in the blob names. The ListBlobs() method returns a list of IListBlobItem objects. They only have a couple of properties: Url and references to parent directory and parent container. I need to get to the metadata of the actual blob objects.

I envisioned there would be a way to either cast the IListBlobItem to a BlockBlob object or use the IListBlockItem to get a reference to the BlockBlob, but can't seem to find a way to do that.

My question is: Is there a way to get a BlockBlob object from this method, or do I have to use a different way of getting the actual BlockBlob objects? If different, then can you suggest a way to achieve this, while also being able to filter by the "directory" scheme?

Painless

Answer 1

OK... I found a way to do this, and while it seems a little clunky and indirect, it does achieve the main thing I thought should be doable, which is to cast the IListBlobItem directly to a CloudBlockBlob object.

What I am doing is getting the list from the Directory object's ListBlobs() method and then looping over each item in the list and casting the item to a CloudBlockBlob object and then calling the FetchAttributes() method to retrieve the properties (including the metadata). Then add a new "info" object to a new list of info objects. Here's the code I'm using:

CloudBlobDirectory dir = container.GetDirectoryReference(dirPath);

var blobs = dir.ListBlobs(true);

foreach (IListBlobItem item in blobs)
{
    CloudBlockBlob blob = (CloudBlockBlob)item;
    blob.FetchAttributes();
    files.Add(new ImageInfo
    {
        FileUrl = item.Uri.ToString(),
        FileName = item.Uri.PathAndQuery.Replace(restaurantId.ToString().PadLeft(3, '0') + "/", ""),
        ImageName = blob.Metadata["Name"]
    });
}

The whole "Blob" concept seems needlessly complex and doesn't seem to achieve what I'd have thought would have been one of the main features of the Blob wrapper. That is, a way to expand search capabilities by allowing a query over name, directory, container and metadata. I'd have thought you could construct a linq query that would read somewhat like: "return a list of all blobs in the 'images' container, that are in the 'natural/landscapes/' directory path that have a metadata key of 'category' with the value of 'sunset'". There doesn't seem to be a way to do that and that seems to be a missed opportunity to me. Oh, well.

If I'm wrong and way off base here, please let me know.

Painless

I completely agree with you that Microsoft's design is utterly silly. I am migrating from MongoDB and at least with MongoDB I could search on metadata. I want to use metadata to flag blobs as having certain interesting properties, then later retrieve all the blobs with those properties. Apparently this is impossible in Windows Azure without iterating each blob or using the blob's name to store filters. – Aristocrat

Answer 2

This approach has been developed for Java, but I hope it can somehow be modified to fit any other supported language. Despite the functionality you ask has not been explicitly developed yet, I think I found a different (hopefully less clunky) way to access CloudBlockBlob data from a ListBlobItem element.

The following code can be used to delete, for example, every blob inside a specific directory.

String blobUri;
CloudBlobClient blobClient = /* Obtain your blob client */

try{
     CloudBlobContainer container = /* Obtain your blob container */

     for (ListBlobItem blobItem : container.listBlobs(blobPrefix)) {

          if (blobItem instanceof CloudBlob) {
                blob = (CloudBlob) blobItem;
                if (blob.exists()){
                    System.out.println("Deleting blob " + blob.getName());
                    blob.delete();
                }
          }
     }
}catch (URISyntaxException | StorageException ex){
        Logger.getLogger(BlobOperations.class.getName()).log(Level.SEVERE, null, ex);
}

Elflock

Answer 3

The previous answers are good. I just wanted to point out 2 things:

1) Nowadays ASYNC programming is recommended to do and supported by Azure SDK as well. So try to use it:

CloudBlobDirectory dir = container.GetDirectoryReference(dirPath);
var blobs = dir.ListBlobs(true);

foreach (IListBlobItem item in blobs)
{
    CloudBlockBlob blob = (CloudBlockBlob)item;
    await blob.FetchAttributesAsync(); //Use async calls...
}

2) Fetching Metadata in a separate call is not efficient. The code makes 2 HTTP request per blob object. ListBlobs() method supports getting Metadata with as well in one call by setting BlobListingDetails parameter:

CloudBlobDirectory dir = container.GetDirectoryReference(dirPath);
var blobs = dir.ListBlobs(useFlatBlobListing: true, blobListingDetails: BlobListingDetails.Metadata);

I recommend to use second code it it is possible. Since it is the most efficient way to fetch Metadata.

Road