Are Delphi DLLs predestined for loader locks?

Question

There is a DLL, that is loaded by the main (desktop) application dynamically via Windows.LoadLibrary. That's because there are plenty of similar DLLs and only few to single of them are required to be loaded on runtime. So static linking is not an option.

The problem is, that every once in a while the main application hangs when loading one of those DLLs. Note, that the problem is likely to happen with every one of them. Probably because they have a lot of codebase in common.

The problem seems to be a loader lock (see this SO answer on what it is). I found a piece of common code, that is used by all of the DLLs at startup in the begin...end-section of the library-unit (that is project.dpr), where GetModuleHandle and GetProcAddress are used.

I found out, that this is a totally DONT with DLLs, as the begin...end-section of the DLL's project file is in fact the library's DllMain function and calling such functions can lead to dead locks (named loader lock). I read that in this Microsoft Best Practice Guide.

So I rebuild my code, that these calls are called later on, when the call of Windows.LoadLibrary has been completed.

Unfortunately, the hanging problem remains. :-(

I ran the debugger then, stepping through every piece of initialization that is called even before a single line of my code is executed. I determined, that lots of third party code contravenes the guide on what to do and what not to do in DLL initialization code:

• TMS Component Package
• JEDI Component Library
• OmniThreadLibrary
• Indy Components

All of the above dynamically load other DLLs in initialization-code or requesting procedure pointers via GetProcAddress. I think that these calls cause the hanging when my DLLs are loaded.

Is it, that only few Delphi developers know about the dangers of initialization? What can I do about this?

Pickpocket

GetModuleHandle() and GetProcAddress() are safe to use during DLL initialization, per the document you linked to. Indy dynamically loads several DLLs at runtime, but none of them should have been loaded in any initialization sections, they are loaded dynamically on an as-needed basis when actually calling DLL functions. But I just found 1 DLL (zlib) that is being loaded in an initialization section, so I have fixed that now. – Agriculture

See the "library" section in the documentation of api functions. GetModuleHandle and GetProcAddress are kernel functions. – Espino

So this site is wrong about GetModuleHandle? It says "This “loader lock” is taken any time a library is loaded but also when functions like GetModuleHandle() or GetModuleFileName() are used." – Segalman

Answer 1

This is a common problem that I don't think is particularly specific to Delphi programmers. If you have code that calls LoadLibrary in an initialization section, or indeed FreeLibrary in a finalization section then that code is not safe to be used in a library.

Note that I am not familiar with all the libraries that you have mentioned, and am not at all confirming that they all have initialization section code that is not safe for use in a library. I think that's something for you to confirm–I'd like to stick to the concepts in this answer rather than comment on specific Delphi libraries.

I would say though that calls to GetModuleHandle and GetProcAddress are fine from DllMain. I say that because you mention GetProcAddress specifically. It's absolutely fine to obtain a module handle by, for instance, calling GetModuleHandle, and then to obtain a function address by calling GetProcAddress. So if some of the suspect libraries do that, and do not call LoadLibrary, then they might be alright.

Anyway, subject to the provisos above, you need to arrange that any code that will be called from DllMain that contravenes the rules laid down by Microsoft is called at a safe time instead, and not from DllMain. Unfortunately those rules are hazy at best. Microsoft say the following in the DllMain documentation:

The entry-point function should perform only simple initialization or termination tasks. It must not call the LoadLibrary or LoadLibraryEx function (or a function that calls these functions), because this may create dependency loops in the DLL load order. This can result in a DLL being used before the system has executed its initialization code. Similarly, the entry-point function must not call the FreeLibrary function (or a function that calls FreeLibrary) during process termination, because this can result in a DLL being used after the system has executed its termination code.

Because Kernel32.dll is guaranteed to be loaded in the process address space when the entry-point function is called, calling functions in Kernel32.dll does not result in the DLL being used before its initialization code has been executed. Therefore, the entry-point function can call functions in Kernel32.dll that do not load other DLLs. For example, DllMain can create synchronization objects such as critical sections and mutexes, and use TLS. Unfortunately, there is not a comprehensive list of safe functions in Kernel32.dll.

The final paragraph leaves you with little guidance. To be confident that your library will be robust you will need to do something along the following lines:

1. Arrange that each initialization section of any unit whose source code you control registers with a central registry an initialization and finalization procedure.
2. In an executable project you call the initialization procedures when they are registered, and call the finalization procedures in reverse order when the program terminates.
3. In a library project you postpone the calling of these initialization and finalization procedures. Export a pair of functions from the DLL that the consumer of the DLL can call to request that these initialization and finalization procedures are called.

This is the approach that I have taken with my libraries and it has served me well for many years.

This approach involves quite a lot of work, and has the downside that you are modifying third party libraries. However, if those libraries will not work correctly when used as delivered, what alternative do you have?

Perhaps in slower time you could contact the developers of any libraries that you believe are not compatible with use in a library. Try to persuade them to change their code such that it is compatible with use in a library. As you can see from Remy's comment to your question, it's entirely possible that the library developers may be unaware of the issue, and very willing to make changes.

Sporran

Answer 2

Idea from Russian blog: http://www.gunsmoker.ru/

You can create Dll in Delphi that does nothing in its DllMain. In order to do that you should create new package like following:

package Plugin1;

//...
{$E dll}

contains
  InitHook,
  //...
  ;

end.

And InitHook:

unit InitHook;

interface

implementation

function GetProcAddress(hModule: HMODULE; lpProcName: PAnsiChar): Pointer; stdcall; external 'kernel32.dll' name 'GetProcAddress';

procedure Done; stdcall;
type
  TPackageUnload = procedure;
var
  PackageUnload: TPackageUnload;
begin
  @PackageUnload := GetProcAddress(HInstance, 'Finalize'); //Do not localize
  PackageUnload;
end;

procedure Init; stdcall;
type
  TPackageLoad = procedure;
var
  PackageLoad: TPackageLoad;
begin
  @PackageLoad := GetProcAddress(HInstance, 'Initialize'); //Do not localize
  PackageLoad;
end;

exports
  Init,
  Done;
end.

Now you can put inside this package any code you wanted to put inside Dll. But you will have to call Init before calling any other function from this dll, and call Done before unloading it.

Initialize and Finalize are procedures, that compiler automatically creates in packages. These procedures execute all initialization and finalization sections in all units in package.

Explicit

Answer 3

If you are doing dynamic loading inside your program, then windows DLLs can't cause loader locks, because they are already loaded when first code in your program gets chance to execute. So loader locks can be caused only between your libraries. In that case you will have to determine correct load order. If you have some documentation, then search in it.

If all your libraries are Delphi/C++ Builder libraries and are compiled in the same RAD Studio release, then I suggest you to enable runtime packages for all of them. This will reduce code duplication and a chance to get lock because two instances of singelton object like Application try to work simultaneously. Or even better - convert your libraries in to packages. This will eliminate any chance for locks.

Explicit

This isn't at all accurate. Some system DLLs are loaded at process start. But by no means all. Others can be loaded later. And load order is not relevant. You have no control over load order of implicitly loaded libraries. Converting to runtime packages hardly helps. They are just DLLs too. And who says that you can even do that. What if you don't control all the DLLs? – Sporran

If you don't believe that converting Delphi Dlls in to packages will 100% eliminate locks between them, then you should better look how they were implemented. – Explicit

As for cases when you have a mix of Delphi and non-Delphi libraries, then you can statically import required non-Delphi libraries to main exe. Or even better: rewrite all Delphi code to C++ and statically link everything. – Explicit

I'm not sure you really understand the rules around the loader lock – Sporran