COM registration fails with error code 0xC0000005

Question

I'm working on a legacy project which contains several C++ COM Dlls. Every time the solution is built on debug configuration, the build process gives errors for each COM project:

Error   10  error MSB3073: The command "
        regsvr32 /s /c "D:\*****removed intentionally****_d.dll"
        echo regsvr32 exec. time > ".\Debug\regsvr32.trg"
        echo Server registration done!
      :VCEnd
    " exited with code -1073741819. 

I'v recently joined this project, so when I asked about it, I've been told that everyone just ignore those errors because the solution builds successfully on the second build.

I decided to dig deeper and it appears that the COM registration itself succeeds (which explains why the 2nd build did not reattempt to register), but the call to regsvr32 returns an error code (0xC0000005). And that's whats failing the build.

I also tried to remove the registration from the custom build step, and instead chose to register from the Linker property sheet "Register Output=YES", and got the same error.

I tried to debug regsvr32 with one of the dlls and found the following things:

1. I couldn't put a breakpoint in DllRegisterServer at first, I then tried to debug the X86 version of regsvr32 (from Windows\SysWOW64), and only then I could break on DllRegisterServer. Can someone explain this?

2. DllRegisterServer succeeds and return S_OK.

3. After DllRegisterServer returns, an exception is thrown with the following exception and stack trace:

First-chance exception at 0x759849c1 in regsvr32.exe: 0xC0000005: Access violation reading location 0x005bf028.

oleaut32.dll!_MemFree@4()  + 0x25 bytes 
oleaut32.dll!OLE_TYPEMGR::Release()  + 0x1b138 bytes    
oleaut32.dll!ReleaseAppData()  + 0x317 bytes    
oleaut32.dll!_OACleanup@0()  + 0x5 bytes    
ole32.dll!wCoUninitialize(COleTls & Tls, int fHostThread)  Line 2830    C++
ole32.dll!CoUninitialize()  Line 2620   C++
regsvr32.exe!_wWinMain@16()  + 0xa77 bytes  
regsvr32.exe!__initterm_e()  + 0x1b1 bytes  
kernel32.dll!@BaseThreadInitThunk@12()  + 0x12 bytes    
ntdll.dll!___RtlUserThreadStart@8()  + 0x27 bytes   
ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes    

So What is your take on this? Should I ignore it like the rest of the team (the product works fine in development and production)?

Do you have an idea why I would get an access violation even though the registration succeeds? is there any other process/logic that runs after the COM dll is registered? any other directions?

This is the code for class that handles the registration:

   CComModule _Module;

BEGIN_OBJECT_MAP(ObjectMap)
OBJECT_ENTRY(CLSID_C3DT2D, CC3DT2D)
END_OBJECT_MAP()

class CMy3DT2DApp : public CWinApp
{
public:

// Overrides
    // ClassWizard generated virtual function overrides
    //{{AFX_VIRTUAL(CMy3DT2DApp)
    public:
    virtual BOOL InitInstance();
    virtual int ExitInstance();
    //}}AFX_VIRTUAL

    //{{AFX_MSG(CMy3DT2DApp)
        // NOTE - the ClassWizard will add and remove member functions here.
        //    DO NOT EDIT what you see in these blocks of generated code !
    //}}AFX_MSG
    DECLARE_MESSAGE_MAP()
};

BEGIN_MESSAGE_MAP(CMy3DT2DApp, CWinApp)
    //{{AFX_MSG_MAP(CMy3DT2DApp)
        // NOTE - the ClassWizard will add and remove mapping macros here.
        //    DO NOT EDIT what you see in these blocks of generated code!
    //}}AFX_MSG_MAP
END_MESSAGE_MAP()

CMy3DT2DApp theApp;

BOOL CMy3DT2DApp::InitInstance()
{
    _Module.Init(ObjectMap, m_hInstance, &LIBID_MY3DT2DLib);
    return CWinApp::InitInstance();
}

int CMy3DT2DApp::ExitInstance()
{
    _Module.Term();
    return CWinApp::ExitInstance();
}

/////////////////////////////////////////////////////////////////////////////
// Used to determine whether the DLL can be unloaded by OLE

STDAPI DllCanUnloadNow(void)
{
    AFX_MANAGE_STATE(AfxGetStaticModuleState());
    return (AfxDllCanUnloadNow()==S_OK && _Module.GetLockCount()==0) ? S_OK : S_FALSE;
}

/////////////////////////////////////////////////////////////////////////////
// Returns a class factory to create an object of the requested type

STDAPI DllGetClassObject(REFCLSID rclsid, REFIID riid, LPVOID* ppv)
{
    return _Module.GetClassObject(rclsid, riid, ppv);
}

/////////////////////////////////////////////////////////////////////////////
// DllRegisterServer - Adds entries to the system registry

STDAPI DllRegisterServer(void)
{
    // registers object, typelib and all interfaces in typelib
    return _Module.RegisterServer(TRUE);
}

/////////////////////////////////////////////////////////////////////////////
// DllUnregisterServer - Removes entries from the system registry

STDAPI DllUnregisterServer(void)
{
    return _Module.UnregisterServer(TRUE);
}

Theologize

Is the DLL an ATL project? – Ogdan

@Ogdan The property "Use of ATL" is set to "Not Using ATL", is that enough to tell that it isn't an ATL project? – Theologize

I think so. Is the DLL a MFC project, then ? – Ogdan

@Ogdan Yes it is. Sorry for not mentioning earlier. "Use of MFC" = "Use MFC in a shared DLL". – Theologize

The DllRegisterServer implementation of the DLL is probably corrupting memory. It may also be the ::InitInstance code. Could you post more infos? – Ogdan

Could it be spoiled Registry? Already registered DLL in question? Can the problem be reproduced on the other machine? – Negate

@Ogdan I'll look into the implementation of DllRegisterServer and InitInstance, and will post more info. – Theologize

@Negate the problem is reproduced on every machine consistently. – Theologize

@Ogdan I've edited the question and added the implementation of DllRegisterServer and InitInstance. So apparently, the dll DOES use ATL. I also read that CComModule is obsolete/deprecated. Could that be causing our issue? – Theologize

Smells like heap corruption – Yorktown

Included all of the relevant code – Theologize

Also note that the same problem affects all COM dlls in the solution, so I would imagine that the problem is with the generic COM server registration mechanism that we use for all dlls... – Theologize

Answer 1

So apparently the culprit was a tool called Visual Leak Detector, which attaches itself when included to a source file.

We discovered it by starting from a fresh ATL project (like manuell suggested) and added the functionality back one step at a time. The moment we added some code that was linked against another dll that includes vld.h, the registration errors returned.

So we removed all references to VLD, and now we are happily registering, even with the original posted code.

We even found this little beauty...

Thanks for the help everyone!

Theologize

Answer 2

I think the memory corruption is trigered by mixing ATL and MFC code for initialization/termination.

If you use VS2010 to generate a "MFC ActiveX Control", you will notice that:

1 The CMy3DT2DApp should derive from COleControlModule, not from CWinApp

2 The InitInstance and ExitInstance method should call COleControlModule::InitInstance and COleControlModule::ExitInstance

3 The DllRegisterServer should be:

AFX_MANAGE_STATE(_afxModuleAddrThis);

if (!AfxOleRegisterTypeLib(AfxGetInstanceHandle(), _tlid))
    return ResultFromScode(SELFREG_E_TYPELIB);

if (!COleObjectFactoryEx::UpdateRegistryAll(TRUE))
    return ResultFromScode(SELFREG_E_CLASS);

return NOERROR;

4 similar implementation for DllUnregisterServer (using AfxOleUnregisterTypeLib)

5 There is no implementation for DllCanUnloadNow

One solution may be to suppress all reference to the CComModule _Module and use regular MFC code.

Another solution would be to let the DLL be a regular ATL Dll.

You should use the VS2010 wizard to show you the two kinds of code base. (When you have created the ATL Dll use the "Add Class" menu option, and select "ATL Control"

Solution greatly depends on the use of MFC and/or ATL functions in the remainding code.

Advice: make the DLL full MFC, suppress the "atl module", and then add support for ATL using Adding ATL support to existing mfc application

Ogdan