Azure crash dumps not appearing

Question

The code below produces a crash dump when I run it in Azure development fabric, but not when I deploy it to the cloud. I have:

• Followed Microsoft's instructions (see below)
• Tested the credentials in Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString by uploading a blob.
• Waited for it to crash (about ten times).
• Stopped the deployment and waited half an hour.

But I still cannot find anything in the storage account. I am targeting .Net 4 using VS 2010 Pro SP1 and deploying using its built-in stuff. What am I doing wrong?

public override void Run()
{
   throw new ApplicationException("bugger");
}

public override bool OnStart()
{
    ServicePointManager.DefaultConnectionLimit = 12;

    DiagnosticMonitorConfiguration config = DiagnosticMonitor.GetDefaultInitialConfiguration();
    string conn_str = RoleEnvironment.GetConfigurationSettingValue("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString");
    CloudStorageAccount account = CloudStorageAccount.Parse(conn_str);
    DiagnosticMonitor diagnosticMonitor = DiagnosticMonitor.Start(account, config);

    CrashDumps.EnableCollection(true);

    return base.OnStart();
}

Here is my configuration:

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="WindowsAzureProject1" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
  <WorkerRole name="WorkerRole1">
    <Imports>
      <Import moduleName="Diagnostics" />
    </Imports>
  </WorkerRole>
</ServiceDefinition>

and

<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="WindowsAzureProject1" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="1" osVersion="*">
  <Role name="WorkerRole1">
    <Instances count="1" />
    <ConfigurationSettings>
      <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="AccountName=XXX;AccountKey=XXX;DefaultEndpointsProtocol=https" />
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

(My real project does produce crash dumps, but I am having trouble analyzing them, which is why I am trying to produce a cut-down example. When that works I will see whether its crash dumps are any better.)

EDIT: Part of the solution is to add

config.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);

which creates the wad-crash-dumps blob container, but unfortunately it remains empty. Also see my question about what happens to diagnostic data when a role fails.

Veliavelick

Answer 1

Added Thread.Sleep(60000) before throwing the exception. I think what was happening is this:

1. Exception thrown.
2. Crash dump saved into "DiagnosticStore" local storage, which is presumably configured with cleanOnRoleRecycle set to false.
3. Virtual machine recycled because instance has died. This terminates the diagnostic manager, aborting any uploads in progress.
4. New instance of diagnostic manager is started. It begins to upload the old dump.
5. Go to 1.

Veliavelick

Answer 2

I believe you are missing the RoleInstanceDiagnosticManager. Try doing this:

DiagnosticMonitorConfiguration config = DiagnosticMonitor.GetDefaultInitialConfiguration();
    string conn_str = RoleEnvironment.GetConfigurationSettingValue("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString");
    CloudStorageAccount account = CloudStorageAccount.Parse(conn_str);
RoleInstanceDiagnosticManager roleInstanceDiagnosticManager = account.CreateRoleInstanceDiagnosticManager(RoleEnvironment.DeploymentId, RoleEnvironment.CurrentRoleInstance.Role.Name, RoleEnvironment.CurrentRoleInstance.Id);

    CrashDumps.EnableCollection(true);
config.Directories.DataSources.Add(AzureLocalStorageTraceListener.GetLogDirectory());
            config.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
config.ConfigurationChangePollInterval = TimeSpan.FromMinutes(5);

            //set the configuration for use
            roleInstanceDiagnosticManager.SetCurrentConfiguration(config);  

RoleEnvironment.Changing += new EventHandler<RoleEnvironmentChangingEventArgs>(RoleEnvironment_Changing);

    return base.OnStart();

Aureaaureate

AFAIK, the call to DiagnosticMonitor.Start() should start the diagnostic monitor. Actually as of SDK 1.3 it should be started automatically before this because I include <Import moduleName="Diagnostics" />. See convective.wordpress.com/2010/12/01/… – Veliavelick

Answer 3

Is it possible you have not set the ScheduledTransferPeriod and LogLevelFilter?

TimeSpan tsLogPeriod = TimeSpan.FromMinutes(double.Parse(RoleEnvironment.GetConfigurationSettingValue("LogIntervalInMinutes")));
            DiagnosticMonitorConfiguration diagnosticMonitorConfiguration = DiagnosticMonitor.GetDefaultInitialConfiguration();

            diagnosticMonitorConfiguration.Logs.ScheduledTransferPeriod = tsLogPeriod;

            diagnosticMonitorConfiguration.Logs.ScheduledTransferLogLevelFilter = LogLevel.Verbose;

            DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", diagnosticMonitorConfiguration);

        }

Hypoderm

ScheduledTransferPeriod was part of the problem (edited into original question last Thursday). The LogLevelFilter should not be relevant because I was having a problem with crash dumps, not traces. Anyway, see my own answer. – Veliavelick