Deadlock when logging of using Simple MAPI TEmail component

Question

We are experiencing an intermittent deadlock using a freeware MAPI/SMAPI implementation. I doubt the implementation is at fault but perhaps changing the logon flags to MapiLogon or a configuration setting on Exchange could resolve this.

Result := MapiLogon(0, LogonProfile, LogonPassword, flLogonFlags, 0, @hSession);

Added cudo's to @J

The use of Simple MAPI altogether is discouraged. The proper action would be to start using Extended MAPI or the Outlook Object Model. While I agree with the statement, I have no influence whatsoever to make that happen.

A solution for the current setup or understanding why the deadlock might occur is still much apreciated.

In short

• Thread 0b60 calls MapiLogof
• During logof, it waits for thread 0894
• Thread 0894 waits for Critical Section 036c
• Critical Section 036c is locked by thread 0b60

Deadlock

the kernel dump shows following critical section being locked and owned by thread b60

    CritSec EMSMDB32!ScStatClose+17ac7 at 354650d0
    WaiterWoken        No
    LockCount          1
    RecursionCount     1
    OwningThread       b60
    EntryCount         0
    ContentionCount    1
    *** Locked

Thread's 0b60 call stack

kernel thread object 88a53758
Note the KeWaitForSingleObject with parameter 87fc3c68 being thread 0894

    b8b4fcec 8093b2e4 87fc3c68 00000006 00000001 nt!KeWaitForSingleObject+0x346 (FPO: [Non-Fpo])
    b8b4fd50 8088b658 00000184 00000000 00000000 nt!NtWaitForSingleObject+0x9a (FPO: [Non-Fpo])
    b8b4fd50 7c82845c 00000184 00000000 00000000 nt!KiSystemServicePostCall (FPO: [0,0] TrapFrame @ b8b4fd64)
    0012f618 7c827b79 77e61d06 00000184 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
    0012f61c 77e61d06 00000184 00000000 00000000 ntdll!NtWaitForSingleObject+0xc (FPO: [3,0,0])
    0012f68c 77e61c75 00000184 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac (FPO: [Non-Fpo])
    0012f6a0 3540fc13 00000184 ffffffff 02102150 kernel32!WaitForSingleObject+0x12 (FPO: [Non-Fpo])
    0012f6b4 3540a226 7c81a1a8 3540546f 02102108 EMSMDB32!XPProviderInit+0x58d5
    0012f6bc 3540546f 02102108 00db29c0 0012f6f0 EMSMDB32!MSProviderInit+0x16af6
    0012f6d0 3553ce40 02102108 35411e97 02102108 EMSMDB32!MSProviderInit+0x11d3f
    00000000 00000000 00000000 00000000 00000000 MSMAPI32!UlRelease+0xe
    /* Reconstructed from MAP file */
    0012f878  00422b81 21B81      mailrequestserver+0x22b81      0001:00021B70       MapiLogoff
    0012f894  00423dde 22DDE      mailrequestserver+0x23dde      0001:00022DB0       TEmail.Logoff

Thread's 0894 call stack is

kernel thread object 87fc3c68
Note the EMSMDB32!ScStatClose call resulting in RtlpWaitOnCriticalSection with parameter 0000036c being the critical section owned by thread 0b60

    0231ff80 7c83d0f7 0000036c 00000004 00000000 ntdll!RtlpWaitOnCriticalSection+0x1a3 (FPO: [Non-Fpo])
    0231ffa0 3544d394 354650d0 00000000 00000001 ntdll!RtlEnterCriticalSection+0xa8 (FPO: [Non-Fpo])
    0231ff98 354650d0                            EMSMDB32!ScStatClose+0x17ac7
    0231ffa4 3544d394                            EMSMDB32!EcUnregisterPushNotification+0x12033
    0231ffa8 354650d0                            EMSMDB32!ScStatClose+0x17ac7
    0231ffb4 3544d114                            EMSMDB32!EcUnregisterPushNotification+0x11db3

Question

1. The call stack shows there's an UnregisterPushNotifications involved. Searching for push notifications, I can't find any reason why we would need that (we just logon, send a mail and logof) but, as this happens entirely in MAPI, I don't know how we can prevent the call from happening.
2. If the push notification can't be ignored/disabled somehow, any pointers to what else might cause/resolve this are very welcome.

Some additional information

1. Both threads belong to the same process
2. MSMAPI32.dll is version 10.0.6861.0
3. EMSMDB32.dll is version 10.0.6742.0

Relevant code from SMapi.pas

function MapiLogoff(lhSession  : LHANDLE;
                    ulUIParam  : ULONG;
                    flFlags    : ULONG;
                    ulReserved : ULONG): ULONG;

function MapiLogon(ulUIParam   : ULONG;
                   lpszName    : PChar;
                   lpszPassword: PChar;
                   flFlags     : ULONG;
                   ulReserved  : ULONG;
                   lplhSession : LPLHANDLE): ULONG;

function MapiSendMail(lhSession  : LHANDLE;
                      ulUIParam  : ULONG;
                      lpMessage  : lpMapiMessage;
                      flFlags    : ULONG;
                      ulReserved : ULONG): ULONG;

procedure InitializeSMAPI;
var
  OldErrorMode: Word;
  OSVersionInfo: TOSVersionInfo;
  RegHandle: HKEY;
  MapiDetectBuf: array[0..8] of Char;
  MapiDetectBufSize: Windows.DWORD;
  RegValueType: Windows.DWORD;
begin
  { first check wether MAPI is available on the system; this is done
    as described in the MS MAPI docs }

  OSVersionInfo.dwOSVersionInfoSize := SizeOf(OSVersionInfo);
  GetVersionEx(OSVersionInfo);
  if (OSVersionInfo.dwMajorVersion > 3) or { NT 4.0 and later }
     { earlier than NT 3.51 }
     ((OSVersionInfo.dwMajorVersion = 3) and (OSVersionInfo.dwMinorVersion > 51)) then
  begin
    if RegOpenKeyEx( HKEY_LOCAL_MACHINE,
                     'SOFTWARE\Microsoft\Windows Messaging Subsystem',
                     0, KEY_READ, RegHandle) <> ERROR_SUCCESS then
    begin
      exit;
    end;

    MAPIDetectBufSize := SizeOf(MAPIDetectBuf);
    if RegQueryValueEx( RegHandle, 'MAPI', nil, @RegValueType,
                        PByte(@MAPIDetectBuf), @MAPIDetectBufSize) <> ERROR_SUCCESS then
    begin
      exit;
    end;

    RegCloseKey(RegHandle);

    { "boolean" integer --> is == "1"? }
    if not ((MAPIDetectBuf[0] = '1') and (MAPIDetectBuf[1] = #0)) then
      exit;
  end
  else
    if GetProfileInt('Mail', 'MAPI', 0) = 0 then { 16 bit and NT 3.51 detection logic }
      Exit;

  OldErrorMode := SetErrorMode(SEM_FAILCRITICALERRORS + SEM_NOOPENFILEERRORBOX);
    DLLHandle := LoadLibrary(DLLName32); { start without .DLL attached }
  { OldErrorMode := } SetErrorMode(OldErrorMode);

  if DLLHandle = 0 then { got an error }
  begin
    OldErrorMode := SetErrorMode(SEM_FAILCRITICALERRORS + SEM_NOOPENFILEERRORBOX);
    try
      DLLHandle := LoadLibrary(DLLName32DLL);

      if DLLHandle = 0 then
      begin
        exit; { second attempt did not work out either }
      end;

    finally
      { OldErrorMode := } SetErrorMode(OldErrorMode);
    end;
  end;
  begin
    DllInitialized := true;

    @FnMapiFindNext :=    GetProcAddress(DLLHandle, 'MAPIFindNext');
    @FnMapiLogoff :=      GetProcAddress(DLLHandle, 'MAPILogoff');
    @FnMapiLogon :=       GetProcAddress(DLLHandle, 'MAPILogon');
    @FnMapiSendMail :=    GetProcAddress(DLLHandle, 'MAPISendMail');
    @FnMapiReadMail :=    GetProcAddress(DLLHandle, 'MAPIReadMail');
    @FnMapiDeleteMail :=  GetProcAddress(DLLHandle, 'MAPIDeleteMail');
    @FnMapiResolveName := GetProcAddress(DLLHandle, 'MAPIResolveName');
    @FnMapiFreeBuffer :=  GetProcAddress(DLLHandle, 'MAPIFreeBuffer');
    @FnMapiAddress :=     GetProcAddress(DLLHandle, 'MAPIAddress');
    @FnMapiSaveMail :=    GetProcAddress(DLLHandle, 'MAPISaveMail');

    if    (@FnMapiAddress     = nil)
       or (@FnMapiFreeBuffer  = nil)
       or (@FnMapiResolveName = nil)
       or (@FnMapiDeleteMail  = nil)
       or (@FnMapiReadMail    = nil)
       or (@FnMapiSendMail    = nil)
       or (@FnMapiLogon       = nil)
       or (@FnMapiLogoff      = nil)
       or (@FnMapiFindNext    = nil)
       or (@FnMapiSaveMail    = nil) then
    begin
      raise EMAPIdllerror.Create(SMapiGetProcAdressFailed);
    end;
  end;
end;

Relevant code from Email.pas

destructor TEmail.Destroy;
begin
  ...
  try
    if hSession <> 0 then
      Logoff;
  except
  end;
end;
function TEmail.Logon: Integer;
const
  ProfileKey95 = 'Software\Microsoft\Windows Messaging Subsystem\Profiles';
  ProfileKeyNT = 'Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem\Profiles';
var
  LogonProfile : PChar;
  LogonPassword: PChar;

  ProfileKey   : PChar;

  Reg : TRegistry;
begin
  CheckMapi;
  Result := SUCCESS_SUCCESS;
  { Check if already logged in. }
  if hSession = 0 then
  begin
    if FUseDefProfile then
    begin
      Reg := TRegistry.Create;
      try
        { get platform (Win95/NT) dependent profile key                }
        {  code added by Ulrik Schoth [email protected] }
        if Reg.KeyExists(ProfileKeyNT) then
        begin
          ProfileKey := ProfileKeyNT;
        end
        else
        begin
          ProfileKey := ProfileKey95;
        end;

        Reg.Rootkey := HKEY_CURRENT_USER;
        if Reg.OpenKey(ProfileKey, False) then
        begin
          try
            FProfile := Reg.Readstring('DefaultProfile');
          except
            FProfile := '';
          end;
        end;
      finally
        Reg.Free;
      end;
    end;

    LogonProfile := nil;
    LogonPassword := nil;

    try
      if Length(FProfile) > 0 then
      begin
        LogonProfile := StrPCopy(StrAlloc(Length(FProfile)+1), FProfile);
      end;

      if Length(FPassword) > 0 then
      begin
        LogonPassword := StrPCopy(StrAlloc(Length(FPassword)+1), FPassword);
      end;

      DoBeforeLogon;

      Result := MapiLogon(0, LogonProfile, LogonPassword, flLogonFlags, 0, @hSession);
      if Result <> SUCCESS_SUCCESS then
        Result := MapiLogon(0, nil, nil, flLogonFlags or MAPI_Logon_UI, 0, @hSession);

      if Result = SUCCESS_SUCCESS then
        DoAfterLogon
      else
        DoMapiError(Result);

    finally
      StrDispose(LogonProfile);
      StrDispose(LogonPassword);
    end;
  end;
end;

function TEmail.SendMailEx(DoSave: boolean): Integer;
var
  MapiMessage   : TMapiMessage;
  MapiRecipDesc : TMapiRecipDesc;
  MapiFileDesc  : TMapiFileDesc;
  lpRecipArray  : TlpRecipArray;
  lpAttachArray : TlpAttachArray;
  lpszPathname  : TlpszPathname;
  lpszFileName  : TlpszFileName;

  szSubject     : PChar;
  szText        : PChar;
  szMessageId   : PChar;
  szMessageType : PChar;

  Attachment    : SString;
  flFlags       : ULONG;
  flLogoff      : Boolean;

  i             : Integer;
  nRecipients   : Integer;
  nAttachments  : Integer;

begin
  CheckMapi;

  {make sure the cleanup does not free garbage }
  lpRecipArray  := nil;
  lpAttachArray := nil;

  flLogoff      := False;

  {check our built-in limits - which have effectively been removed }
  nRecipients := Frecip.Count + FCC.Count + FBCC.Count;
  if nRecipients > RECIP_MAX then
  begin
    Result := MAPI_E_TOO_MANY_RECIPIENTS;

    DoMapiError(Result);

    exit;
  end;

  nAttachments := FAttachment.Count;
  if nAttachments > ATTACH_MAX then
  begin
    Result := MAPI_E_TOO_MANY_FILES;

    DoMapiError(Result);

    exit;
  end;

  { begin the work }
  try

    flLogoff := (hSession = 0);

    { Logon to mail server if not already logged on. }

    if Logon <> SUCCESS_SUCCESS then
    begin
      Result := MAPI_E_LOGIN_FAILURE;

      DoMapiError(Result);

      exit;
    end;


    { Initialise MAPI structures and local arrays. }

    FillChar(MapiMessage,   SizeOf(TMapiMessage),   0);
    FillChar(MapiRecipDesc, SizeOf(TMapiRecipDesc), 0);
    FillChar(MapiFileDesc,  SizeOf(TMapiFileDesc),  0);

    lpRecipArray  := TlpRecipArray(StrAlloc(nRecipients*SizeOf(TMapiRecipDesc)));
    FillChar(lpRecipArray^, StrBufSize(PChar(lpRecipArray)), 0);

    lpAttachArray := TlpAttachArray(StrAlloc(nAttachments*SizeOf(TMapiFileDesc)));
    FillChar(lpAttachArray^, StrBufSize(PChar(lpAttachArray)), 0);

    { Fill in subject & message text. }

    szSubject      := nil;
    szText         := nil;
    szMessageId    := nil;
    szMessageType  := nil;

    try
      if Length(FSubject) > 0 then
      begin
        szSubject := StrAlloc(length(FSubject) + 1);
        StrPCopy(szSubject, FSubject);
      end;
      MapiMessage.lpszSubject  := szSubject;

      if Length(FText) > 0 then
      begin
        szText := StrAlloc(length(FText) + 1);
        StrPCopy(szText, FText);
      end;
      MapiMessage.lpszNoteText := szText;

      { for non-IPM messages }
      if Length(FMessageType) > 0 then
      begin
        szMessageType := StrAlloc(Length(FMessageType) + 1);
        StrPCopy(szMessageType, FMessageType);
      end;
      MapiMessage.lpszMessageType := szMessageType;

      if FpLongText <> nil then
        MapiMessage.lpszNoteText := FpLongText;

      { check and fill in recipients if any}
      nRecipients := 0;
      ListToRecipArray(FRecip, MAPI_TO,  lpRecipArray, nRecipients);
      ListToRecipArray(FCC,    MAPI_CC,  lpRecipArray, nRecipients);
      ListToRecipArray(FBcc,   MAPI_BCC, lpRecipArray, nRecipients);


      MapiMessage.nRecipCount := nRecipients;

      flFlags := 0; { Don't display MAPI Dialog if recipient specified. }

        MapiMessage.lpRecips := @lpRecipArray^;

      { Process file attachments. }

      nAttachments := 0;

      for i := 0 to (Fattachment.Count - 1) do
      begin
        Attachment := CheckAttachment(Fattachment.Strings[i]);
        if Length(Attachment) = 0 then
        begin
          Result := MAPI_E_ATTACHMENT_NOT_FOUND;

          DoMapiError(Result);

          exit;
        end;
        lpAttachArray^[i].nPosition    := Integer($FFFFFFFF);  {Top of message. }
        lpszPathname                   := new(TlpszPathname);
        lpAttachArray^[i].lpszPathName := StrPcopy(lpszPathname^, Attachment);


      { begin code added by MJK }
        lpszFileName                   := new(TlpszFileName);

        { truncate attachment filename if desired }
        if FTruncAttFN then
        begin
          { truncate }
          lpAttachArray^[i].lpszFileName :=
            StrPCopy(lpszFileName^, TruncAttachmentFN(ExtractFileName(Attachment)))
        end
        else
        begin
          { leave alone }
          lpAttachArray^[i].lpszFileName :=
            StrPCopy(lpszFileName^, ExtractFileName(Attachment));
        end;
      {end code added by MJK}

        Inc(nAttachments);
      end;

      MapiMessage.nFileCount := nAttachments;
      if nAttachments > 0 then
      begin
        MapiMessage.lpFiles := @lpAttachArray^;
      end
      else
      begin
        MapiMessage.lpFiles := nil;
      end;

      { receipt requested ? }
      if FAcknowledge then
        MapiMessage.flFlags := MapiMessage.flFlags or MAPI_RECEIPT_REQUESTED;

      { finally send the email message }

      DoBeforeSendMail;

      Result := MapiSendMail(hSession, 0, @MapiMessage, flFlags, 0);
      if Result = SUCCESS_SUCCESS then
        DoAfterSendMail
      else
        DoMapiError(Result);


    finally
      StrDispose(szSubject);
      StrDispose(szText);
      StrDispose(szMessageID);
      StrDispose(szMessageType);
    end;

  finally
    { dispose of the recipient & CC name strings }
    if Assigned(lpRecipArray) then
      for i := 0 to (nRecipients - 1) do
      begin
        if Assigned(lpRecipArray^[i].lpszName) then
          Dispose(lpRecipArray^[i].lpszName);

        if Assigned(lpRecipArray^[i].lpszAddress) then
          Dispose(lpRecipArray^[i].lpszAddress);
      end;

    { dispose of the recipient/CC/BCC array }
    StrDispose(PChar(lpRecipArray));

    { dispose of the attachment file name strings }
    if Assigned(lpAttachArray) then
      for i := 0 to (nAttachments - 1) do
      begin
        Dispose(lpAttachArray^[i].lpszPathname);
        Dispose(lpAttachArray^[i].lpszFileName);
      end;

    { dispose of the attachment array }
    StrDispose(PChar(lpAttachArray));

    { Auto logoff, if no session was active. }
    if flLogoff = True then
      Logoff;
  end;

end;

ps. I can't post the dump, it's 4GB but feel free to ask any additional required details I might have left out

Tutor

It would seem that this freeware component was born in the age of Win3.1 and hasn't seen any development in the past 16 years... you might rather consider moving to a more modern solution. – Breccia

@J - The component is old but the culprit I believe is MSMAPI32.dll/EMSMDB32.dll. The age of the component as such doesn't matter, a new component would still have to use mapilogon, mapisendmail and mapilogof from MSMAPI32.dll. – Tutor

@whosrdaddy - My primary suspect is the EMSMDB32.dll. I assume it starts a new thread to do the registration and whatnot. The error is not in my code but perhaps by initializing the MAPI session in another way, the EMSMDB32 doesn't need to register and doesn't result in a deadlock. (or perhaps even by changing an Exchange parameter) – Tutor

I seriously doubt that those DLLs are the problem - they are extremely mature and have been in heavy use for years. I would have a look at email32.pas that provides the TEmail component - surely the calling code is at fault here. – Breccia

@J - The CS owned by my main thread isn't created by our code, I assume its created by EMSMDB32. The second thread isn't created by our code either, I assume it's also created by EMSMDB32 (entry point certainly seems to confirm that). I have already looked over the code in email32.pas and couldn't find anything suspicious. I'll post some relevant code if you wait a bit (at home now, have to remote in) – Tutor

You don't have to own the CS to do something illegal - corrupt a session, etc. Who knows, it could also be a DLL versioning conflict between whatever versions of MAPI32.DLL have changed since 1998. MSDN says The use of this function is discouraged. It may be altered or unavailable in subsequent versions of Windows. msdn.microsoft.com/en-us/library/dd296726%28v=vs.85%29.aspx – Breccia

@J-I have added the code that initializes, logs on, sends and logs of from MAPI. I for one hope you are right it's an error in our code. We can fix our code :) – Tutor

Tx for the link but following that, it seems that the entire use of Simple MAPI is discouraged. If that's so, do you know what alternatives we have? – Tutor

Generally you would use either Extended MAPI or Outlook Object Model to do this sort of thing now. – Breccia

It's very unlikely that the app is going to be rewritten but rather it being phased out. If you know any pointers that migth lead to a solution for the given app, be it either exchange configuration, client setup or our custom code, by all means, shoot <g> – Tutor

Answer 1

I have had the opportunity to get the dump analyzed by a friendly party.

Conclusions are (my interpretation)

• This could be a bug in EMSMDB32.dll but the code is so old that there's more than likely not any support or intention to fix it.
• The offset in EMSMDB32!EcUnregisterPushNotification+0x11db3 is so large that it's likely not executing the EcUnregisterPushNotification method. What method instead might be executing could not be determined.

Possible solutions

As it's impossible for us to fix the bug (assuming it is a bug), following workarounds would work in our case

• Restart the service every x minutes.
• Group the mails to send by profile and send as batch instead of logging on/off for each mail.
• Start a secondary thread to watch the main thread and have it restart the service if a deadlock is detected.

Tutor