Unknown Disk Read Bytes and Disk Operations at Azure Ubuntu VM service after that VM is not responding
Asked Answered
A

1

6

Each week nearly two or three times, azure vm stops the responding. When it happens, Disk Read Bytes are dramatically increasing i dont know why. My application has no read disk function (only writes the logs). As you see in the figure, Read Bytes and Disk Operations are increasing for a moment that causes server to freeze. I have to do restart the VM and it takes nearly 15 minutes to be available again.

Azure Ubuntu VM Stats

i am using Azure Ubuntu 16.04 VM and Size is Standard B1s (1 vcpus, 1 GiB memory).

i have alredy checked these conditions

  • possible memory leaks by my application
  • observed all source codes of application whether disk read function exists or not (i dont have any disk ready operation at my application)
  • i have changed default virtual ram size at ubuntu %60 to %1
  • i am using docker to run my application and i am using only 1 instance of docker image at vm
  • i removed all vm and i created it again (including all resources i.e. disk/network etc.)

i want to know why this is happening and how can i investigate what are cause this problem. I haven't seen any suspicious running process at ubuntu when it is happening.

Allie answered 12/7, 2019 at 21:28 Comment(4)
Have you attempted to increase the size of your VM to see if that helps? The B series is burstable so performance can vary. I would be curious to see if you picked a more consistent size maybe 2vCPU and 3.5GB of ram if the issue still persisted.Warfield
i am not using my all resoruces as you see, cpu/ram and disk operations. average cpu %10, RAM usage apprx. %85 ,disk operations much more lower than %1. i have tested my work at local VM and different hostings(nearly same size).I didn't test for a higher size in Azure. Even if it works, the reason it won't take the size of the VM is the waste of resources. I am curious what causes that peak at Disk Read Bytes and Disk Operation Graphics.Allie
Thanks for that. I believe it would be best to have this looked into deeper by Azure Support. Do you have the ability to open technical support ticket? If not, you can email me at [email protected] and provide me with your Azure SubscriptionID and link to this thread. I can then enable you for a free support request.Warfield
i don't able to open ticket. i sent a mail from [email protected]Allie
A
6

This issue is solved by Azure Technical Support Engineers (Thanks to Micah_MSFT)

That seems Azure VM is going to out of memory. When it happens, VM is acting unsteady, so Disk Operations is rapidly increasing and VM is freezing (they are investigating the issue's details and reported to Linux Technicians).

As clarified by Azure Technical Support Engineer, There is a feature named Swap Space which helps the saver to reserve storage space as processing memory.

Here is the configuration

  1. Open the file located under /etc/waagent.conf find these variables and modify them accordingly:
# Format if unformatted. If 'n', resource disk will not be mounted.​
ResourceDisk.Format=n​   /// change this parameter to "y"

# Create and use swapfile on resource disk.​
ResourceDisk.EnableSwap=n​   /// change this parameter to "y"
​
# Size of the swapfile.​
ResourceDisk.SwapSizeMB=0​   /// change this parameter to "1024"
  1. Restart the service to apply the changes

sudo systemctl restart walinuxagent.service

That's it

In my opinion, unnecessary increase in disk usage as a result of Out of Memory limit will cause Azure VMs performance to decrease.

Allie answered 20/7, 2019 at 14:39 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.