Memory metrics missing from Azure dashboard

Question

We've recently started using Azure to host some virtual machines, but I've got problems getting the grips on the available resource monitoring metrics.

When I go to the dashboard for the virtual machine, I have the option to add metrics for several things, but Memory Available is missing:

When reading about how to monitor cloud services, it seems clear that you should have the option to add a metrics for Memory Available. Reading other posts here on Stack Overflow, I see other tools such as MetricsHub mentioned - but I don't think this is what we want, as we don't need any monitoring endpoint, we only want to see memory usage in the Azure dashboard (and apps from the Azure store isn't available to us, since we're on an Enterprise Agreement).

Am I missing something obvious here? What must be done to add memory monitoring to the dashboard?

Accessible

Cloud Services is not the same as Virtual Machines. When you use cloud services, Azure will provision VMs for you and Azure is able to install monitoring tools that see the amount of available memory. When you create your own VMs Azure can't and shouldn't do that. In other words, with VMs you are on your own. The metrics you do see in the portal are the ones that can be measured from outside the VM. – Diphenylhydantoin

Thanks @Frans. If you put this up an an answer, I'll accept it to give you credit for it. One final question though: each of my VMs are backed by its own cloud service. And if I in the management portal go to Cloud Services > [select_one_of_the_cloud_services] > Monitor, I still only see the same metrics as for the VM. Does that mean the the kind of metrics available for a cloud service depends on what is running on the service? – Accessible

Answer 1

Cloud Services is not the same as Virtual Machines. When you use cloud services, Azure will provision VMs for you and Azure is able to install monitoring tools that see the amount of available memory. When you create your own VMs Azure can't and shouldn't do that. In other words, with VMs you are on your own. The metrics you do see in the portal are the ones that can be measured from outside the VM.

If you do deploy as a Cloud Service then initially you will only have the same metrics as for the VM. There are several ways you can change this.
The easiest is to go to the configuration for your cloud service in the Management Portal and change the logging level from Minimal to Verbose; That will enable a lot more metrics. Alternatively, you can specify which metrics you want collected in the cloud configuration in your project in Visual Studio. It is also possible to do this in code, though that is not the currently recommended practice, instead use the configuration tool in the cloud project in visual studio.

The key thing to understand about the metrics in Cloud Services is that, whichever way you elect to configure them, they are stored in a standard way in Table Storage and Blob Storage. That means using the Azure Management Portal or the tool in Visual Studio or code, the outcome is the same. This also means that a variety of tools including Cerebrata, Visual Studio and, indeed, the management portal can all read this data. It is also worth noting that because of the way this works, the configuration can be changed at runtime, usually through the portal but there are other tools and approaches in code.

In my experience, you normally only want to sample your performance metrics every two minutes, but do the log shipping every minute. Also note that you can configure trace logs and IIS logs etc to be available to tools like Visual Studio and Cerebrata. For Cloud Services, it is quite rich functionality but it takes some working with it before you start to "get" it all. Enjoy!

Diphenylhydantoin

Thanks a lot for the in-depth reply, that was really helpful. – Accessible

Thanks. I switched from minimal to verbose logging on my Azure web role and it immediately added memory available to the monitor tab graph. – Toh

If I change it in the portal, will it be overwritten the next time I do a deploy? – Nottinghamshire

I can't find the minimal/verbose option in the new portal. Is that because it doesn't exist, or am I just looking in the wrong place? – Penis

The metrics you do see in the portal are the ones that can be measured from outside the VM. This is not correct. Hyper-V supports dynamic memory allocation so it definitely does know how much memory the guest is using. – Coffeng

Answer 2

You can monitor memory and other "Guest" level metrics in Azure, here's how:

1. in Azure, go to your virtual machine, scroll down the settings to Monitoring > Diagnostics Settings
2. Click to enable Guest level monitoring, it can take a few minutes
3. Then you can go into Metrics for the VM, or Monitor at the top level:
   • choose the resource (the VM)
   • choose Guest in the metric namespace, it will load all the new metrics
   • choose Memory\Committed bytes or whatever ones you want.
4. You can then pin to dashboard etc as you would normally

Trademark

Answer 3

It should be possible to install azure diagnostics on VM using powershell command Set-AzureVMDiagnosticsExtension

http://azure.microsoft.com/en-us/documentation/articles/cloud-services-dotnet-diagnostics/#virtual-machine

or using new management portal

http://feedback.azure.com/forums/231545-diagnostics-and-monitoring/suggestions/5535368-provide-azure-diagnostics-runtime-for-vm-iaas

I've tried to configure it using new portal, I can see the the extension IaaSDiagnostics is installed on VM, but no luck yet with getting the data.

Transpadane