Azure Blobs billing break down of "All Other Operations"

Question

I am looking at the billing for my Azure Storage Account and trying to understand managing its cost.

Currently my blobs cost is mostly under the "All Other Operations" category. Is there a way to see what operations these are?

I would like to reduce this cost, so the goal is to update my app so these operations are performed less, but I need to first identify what they are.

Below is the graph I get from cost analysis. (Storage accounts, Accumulated cost, grouped by meter)

Ricketts

Answer 1

After a support call with Azure, they pointed me to some of the (somewhat hidden) tracing capabilities.

First and easiest is to check the type of transactions.

• Go to the storage account > Metrics
• Select Transactions as the metric
• Click Add Filter and select API Name as the property
• Select the API names you think are the suspects

Unfortunately selecting multiple doesn't show them separately, so you have to try every API individually and see if anything sticks out.

---

Second option is to enable Diagnostics Logging for the storage type you're interested in.

If the above doesn't yield any good results, or you're curious about the exact calls at exact times .etc. you can enable this feature, and wait for logs to be collected, usually over a few days, so you have a good sample set to reason.

• Go to the storage account > Diagnostic settings (classic).
  • This is under Monitoring (classic) doesn't seem to have a replacement in the new Monitoring section.
• Enable logging and metrics type (hour or minute)
• Click Save

These logs are written to a blob storage in the same account to a container named $logs. According to docs, this container cannot be deleted after enabling, but the content can be deleted when you're done.

Note that if your storage account gets a lot of traffic, this log can get very big very quick. You're charged the same rates for reads, writes and storage in this container as usual, including the log writes the platform does when these settings are enabled.

See documentation here

After setting this up, give it some time to collect data.

Use storage explorer or other means to navigate and download the logs and inspect them.

Logs contain every request made to storage, with details such as timestamp, API name, result, whether the operation was authenticated, and if you're looking at blobs it also shows the url, the user agent and more.

(turns out my app did close to 100,000 calls to GetBlobProperties and GetContainerProperties per day🎈)

Ricketts

Wow - that's pretty convoluted, but it does the trick! Thanks for posting. – Heartsome

In my example the "lifecycle management" feature started to issue 1000s of requests. It has a very poor design, it scans all blobs regardless of the prefix. – Mascia

Do you know what caused the 100,000 calls to GetBlobProperties? I'm seeing a very similar thing happening, and I'm wondering if it might be function apps with blob triggers. Thank you for posting! – Sethsethi

This was a while back, but the root cause for me was my code. You can point Azure Functions to a separate storage account to see if its your code or the runtime. If it is not the runtime, you'll have to instrument your code to see where the issue is. I recommend wrapping all calls to the blobs in a service, and log to app insights (add custom dimensions to the logs so you can filter), or similar. You can also enable dependency tracking in App Insights but that likely wont tell you much you don't already know. Either way, this will likely flood appinsights, so beware of the associated costs. – Ricketts

Answer 2

Short answer to your question is Yes.

Analysis: According to my observation, I get "all other operations" when I group by "Meter" as shown in below screenshot.

And then if I export the results by clicking on "Export" and then when I filter the results for "Meter" column with "all other operations" then I observe that column named "ServiceTier" has "tiered block blob" as value (in my case). For reference, see below screenshot.

And then if I group by "Meter subcategory" as shown in below screenshot then I see "tiered block blob" (in my case).

And then if I export the results by clicking on "Export" and then when I filter the results for "Meter subCategory" column with "tiered block blob" then I observe that column named "ServiceTier" also has "tiered block blob". For reference, see below screenshot.

So based on above analysis, I believe that we can figure out the break down of "Meter" column with "all other operations" as "tiered block blob" in my case with the help of "Meter subcategory" and "ServiceTier". Similarly you would be able to figure out the break down of "Meter" column with "all other operations".

Other related references: As per this and this Azure documents, there are many other operations on blobs excluding write, read, list operations so in your case any such operations might have fallen under "all other operations" category.

Shalne

Yea, I see the same. But it doesn't answer which operations I should be looking to reduce to minimize that expense. – Ricketts

and so... what operation is the "tiered block blob" kind? – Inbound