Discover missing module using command-line ("DLL load failed" error)

Question

On Windows, when we try to import a .pyd file, and a DLL that the .pyd depends on cannot be found, we get this traceback:

Traceback (most recent call last):
  ...
ImportError: DLL load failed: The specified module could not be found.

When this happens, often one has to resort to a graphical tool like Dependencies to figure out what is the name of the missing module.

How can I obtain the missing module name via the command-line?

Context: often we get this error in CI, and it would be easier to login via SSH to find out the missing module name, rather than having to log via GUI.

Hanhhank

Answer 1

First, let's chose a concrete example: NumPy's _multiarray_umath*.pyd (from Python 3.9 (pc064)). Note that I'll be reusing this console:

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q074877580]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts\python.exe"
Python 3.9.9 (tags/v3.9.9:ccb0e6a, Nov 15 2021, 18:08:50) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import os
>>>
>>> os.getpid()
12788
>>>
>>> from numpy.core import _multiarray_umath as _mu
>>>
>>> _mu
<module 'numpy.core._multiarray_umath' from 'e:\\Work\\Dev\\VEnvs\\py_pc064_03.09_test0\\lib\\site-packages\\numpy\\core\\_multiarray_umath.cp39-win_amd64.pyd'>
>>> ^Z


[prompt]>
[prompt]> :: Backup %PATH%
[prompt]> set _PATH=%PATH%

To make things as generic as possible, that .pyd depends on a custom .dll (OpenBLAS):

Here's a snapshot of the above (Python) process:

Notice where the dependent .dll was loaded from (2 rows below our (selected) .pyd).

Now, back to the question: there are a bunch of tools that can do that.
But it's important to mention that no matter what tool you use, will (most likely) depend on the PATH environment variable contents (in the 1^st image, the dependent .dll (and others) was not found). Check [MS.Learn]: Dynamic-Link Library Search Order for more details about .dlls.

As a note, since (some) tools generate a lot of output, I'll be filtering it out (using commands like FindStr (Grep)), only showing the relevant parts, in order to avoid filling the answer with junk.

1. [GitHub]: lucasg/Dependencies

Besides the GUI application that you mentioned (DependenciesGui.exe), there's a command line tool as well next to it: Dependencies.exe:

[prompt]>
[prompt]> :: Restore %PATH%
[prompt]> set PATH=%_PATH%

[prompt]>
[prompt]> "f:\Install\pc064\LucasG\DependencyWalkerPolitistTexan\Version\Dependencies.exe" -h
Dependencies.exe : command line tool for dumping dependencies and various utilities.

Usage : Dependencies.exe [OPTIONS] <FILE>

Options :
  -h -help : display this help
  -json : activate json output.
  -cache : load and use binary cache in order to prevent dll file locking.
  -depth : limit recursion depth when analysing loaded modules or dependency chain. Default value is infinite.
  -apisets : dump the system's ApiSet schema (api set dll -> host dll)
  -apisetsdll : dump the ApiSet schema from apisetschema <FILE> (api set dll -> host dll)
  -knowndll : dump all the system's known dlls (x86 and x64)
  -manifest : dump <FILE> embedded manifest, if it exists.
  -sxsentries : dump all of <FILE>'s sxs dependencies.
  -imports : dump <FILE> imports
  -exports : dump <FILE> exports
  -modules : dump <FILE> resolved modules
  -chain : dump <FILE> whole dependency chain

[prompt]>
[prompt]> "f:\Install\pc064\LucasG\DependencyWalkerPolitistTexan\Version\Dependencies.exe" -modules "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Lib\site-packages\numpy\core\_multiarray_umath.cp39-win_amd64.pyd" | findstr "libopenblas"
[NOT_FOUND] libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll :

[prompt]>
[prompt]> set PATH=%_PATH%;e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Lib\site-packages\numpy\.libs

[prompt]>
[prompt]> "f:\Install\pc064\LucasG\DependencyWalkerPolitistTexan\Version\Dependencies.exe" -modules "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Lib\site-packages\numpy\core\_multiarray_umath.cp39-win_amd64.pyd" | findstr "libopenblas"
[Environment] libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll : e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Lib\site-packages\numpy\.libs\libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll

Side note - as seen in [SO]: How to run a fortran skript with ctypes? (@CristiFati's answer), sometimes (for some reason unknown to me) it doesn't show the exports (the GUI, at least).

2. Dependency Walker

Although it's no longer maintained, it's a very nice tool and before Dependencies it was the best I could find. I also used it for [SO]: How to build a DLL version of libjpeg 9b? (@CristiFati's answer) (somewhere at the end).
The drawback is that it's spitting the output in a file, so an additional step is required:

[prompt]>
[prompt]> :: Restore %PATH%
[prompt]> set PATH=%_PATH%

[prompt]>
[prompt]> dir /b

[prompt]>
[prompt]> :: Help not available in console (/? will open GUI)
[prompt]>
[prompt]> "c:\Install\pc064\Depends\DependencyWalkerPolitistTexan\Version\depends.exe" /c /ot:_mu0.txt "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Lib\site-packages\numpy\core\_multiarray_umath.cp39-win_amd64.pyd"

[prompt]> type _mu0.txt | findstr -i "libopenblas"
     [ ? ] LIBOPENBLAS.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.GFORTRAN-WIN_AMD64.DLL
[ ? ]  LIBOPENBLAS.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.GFORTRAN-WIN_AMD64.DLL                                       Error opening file. The system cannot find the file specified (2).

[prompt]>
[prompt]> set PATH=%_PATH%;e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Lib\site-packages\numpy\.libs

[prompt]>
[prompt]> "c:\Install\pc064\Depends\DependencyWalkerPolitistTexan\Version\depends.exe" /c /ot:_mu1.txt "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Lib\site-packages\numpy\core\_multiarray_umath.cp39-win_amd64.pyd"

[prompt]> type _mu1.txt | findstr -i "libopenblas"
     [  6] e:\work\dev\venvs\py_pc064_03.09_test0\lib\site-packages\numpy\.libs\LIBOPENBLAS.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.GFORTRAN-WIN_AMD64.DLL
[  6]  e:\work\dev\venvs\py_pc064_03.09_test0\lib\site-packages\numpy\.libs\LIBOPENBLAS.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.GFORTRAN-WIN_AMD64.DLL  2022/11/30 14:57  2022/11/20 00:44  35,695,412  A      0x0220BC27     0x0220BC27     x64  Console    None        0x00000000622C0000  Unknown      0x01E88000    Not Loaded  N/A              N/A              0.0        2.30        4.0     5.2

3. [MS.Learn]: DUMPBIN Reference

Part of VStudio. I am only listing it as a reference, because it can display a .dll dependents, but not whether they can be loaded (and if yes, where from):

[prompt]>
[prompt]> :: Restore %PATH%
[prompt]> set PATH=%_PATH%

[prompt]>
[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Auxiliary\Build\vcvarsall.bat" x64 > nul

[prompt]>
[prompt]> dumpbin /DEPENDENTS "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Lib\site-packages\numpy\core\_multiarray_umath.cp39-win_amd64.pyd"
Microsoft (R) COFF/PE Dumper Version 14.29.30147.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Lib\site-packages\numpy\core\_multiarray_umath.cp39-win_amd64.pyd

File Type: DLL

  Image has the following dependencies:

    libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll
    python39.dll
    KERNEL32.dll
    VCRUNTIME140.dll
    api-ms-win-crt-math-l1-1-0.dll
    api-ms-win-crt-heap-l1-1-0.dll
    api-ms-win-crt-stdio-l1-1-0.dll
    api-ms-win-crt-string-l1-1-0.dll
    api-ms-win-crt-environment-l1-1-0.dll
    api-ms-win-crt-runtime-l1-1-0.dll
    api-ms-win-crt-convert-l1-1-0.dll
    api-ms-win-crt-time-l1-1-0.dll
    api-ms-win-crt-utility-l1-1-0.dll
    api-ms-win-crt-locale-l1-1-0.dll

  Summary

       40000 .data
       18000 .pdata
       64000 .rdata
        3000 .reloc
        1000 .rsrc
      1F3000 .text

4. Nix emulators

Invoke [Man7]: LDD(1).
I guess this can be a favorite, since it's leaning towards Nix world (where these kind of things are easier) and you also mentioned SSH connection.
I'll be exemplifying on MSYS2, but same thing is achievable from others (Cygwin, maybe MinGW, ...).

[cfati@cfati-5510-0:/e/Work/Dev/StackOverflow/q074877580]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[064bit prompt]>
[064bit prompt]> uname -a
MSYS_NT-10.0-19045 cfati-5510-0 3.4.3-dirty.x86_64 2022-12-19 20:20 UTC x86_64 Msys
[064bit prompt]>
[064bit prompt]> _PATH="${PATH}"
[064bit prompt]>
[064bit prompt]> ls "/e/Work/Dev/VEnvs/py_pc064_03.09_test0/Lib/site-packages/numpy/core/_multiarray_umath.cp39-win_amd64.pyd"
/e/Work/Dev/VEnvs/py_pc064_03.09_test0/Lib/site-packages/numpy/core/_multiarray_umath.cp39-win_amd64.pyd
[064bit prompt]>
[064bit prompt]> ldd "/e/Work/Dev/VEnvs/py_pc064_03.09_test0/Lib/site-packages/numpy/core/_multiarray_umath.cp39-win_amd64.pyd"
ldd: /e/Work/Dev/VEnvs/py_pc064_03.09_test0/Lib/site-packages/numpy/core/_multiarray_umath.cp39-win_amd64.pyd: Bad file descriptor
[064bit prompt]>
[064bit prompt]> # Change extension
[064bit prompt]> cp "/e/Work/Dev/VEnvs/py_pc064_03.09_test0/Lib/site-packages/numpy/core/_multiarray_umath.cp39-win_amd64.pyd" ./_mu.dll
[064bit prompt]> ls
_mu.dll  _mu0.txt  _mu1.txt
[064bit prompt]> file _mu.dll
_mu.dll: PE32+ executable (DLL) (GUI) x86-64, for MS Windows
[064bit prompt]>
[064bit prompt]> ldd _mu.dll
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ff8ba930000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ff8ba320000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ff8b8070000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ff8b8f40000)
        _mu.dll => /e/Work/Dev/StackOverflow/q074877580/_mu.dll (0x7ff86ab50000)
        ucrtbase.dll => /c/Windows/System32/ucrtbase.dll (0x7ff8b8840000)
        vcruntime140.dll => /c/Windows/System32/vcruntime140.dll (0x7ff8a0980000)
        libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll => not found
        python39.dll => not found
        api-ms-win-crt-math-l1-1-0.dll => not found
        api-ms-win-crt-heap-l1-1-0.dll => not found
        api-ms-win-crt-stdio-l1-1-0.dll => not found
        api-ms-win-crt-string-l1-1-0.dll => not found
        api-ms-win-crt-environment-l1-1-0.dll => not found
        api-ms-win-crt-runtime-l1-1-0.dll => not found
        api-ms-win-crt-convert-l1-1-0.dll => not found
        api-ms-win-crt-time-l1-1-0.dll => not found
        api-ms-win-crt-utility-l1-1-0.dll => not found
        api-ms-win-crt-locale-l1-1-0.dll => not found
[064bit prompt]>
[064bit prompt]> PATH="${_PATH}:/e/Work/Dev/VEnvs/py_pc064_03.09_test0/Lib/site-packages/numpy/.libs"
[064bit prompt]>
[064bit prompt]> ldd _mu.dll
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ff8ba930000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ff8ba320000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ff8b8070000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ff8b8f40000)
        _mu.dll => /e/Work/Dev/StackOverflow/q074877580/_mu.dll (0x7ff86ab50000)
        ucrtbase.dll => /c/Windows/System32/ucrtbase.dll (0x7ff8b8840000)
        vcruntime140.dll => /c/Windows/System32/vcruntime140.dll (0x7ff8a0980000)
        libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll => /e/Work/Dev/VEnvs/py_pc064_03.09_test0/Lib/site-packages/numpy/.libs/libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll (0x622c0000)
        user32.dll => /c/Windows/System32/user32.dll (0x7ff8b92f0000)
        win32u.dll => /c/Windows/System32/win32u.dll (0x7ff8b8940000)
        gdi32.dll => /c/Windows/System32/gdi32.dll (0x7ff8b94a0000)
        gdi32full.dll => /c/Windows/System32/gdi32full.dll (0x7ff8b8400000)
        msvcp_win.dll => /c/Windows/System32/msvcp_win.dll (0x7ff8b8610000)
        python39.dll => not found
        api-ms-win-crt-math-l1-1-0.dll => not found
        api-ms-win-crt-heap-l1-1-0.dll => not found
        api-ms-win-crt-stdio-l1-1-0.dll => not found
        api-ms-win-crt-string-l1-1-0.dll => not found
        api-ms-win-crt-environment-l1-1-0.dll => not found
        api-ms-win-crt-runtime-l1-1-0.dll => not found
        api-ms-win-crt-convert-l1-1-0.dll => not found
        api-ms-win-crt-time-l1-1-0.dll => not found
        api-ms-win-crt-utility-l1-1-0.dll => not found
        api-ms-win-crt-locale-l1-1-0.dll => not found

Of course, there can be more tools that I am not aware of (or if .dlls coming from .NET are involved).

Related (more or less):

• [SO]: Can't import dll module in Python (@CristiFati's answer)

• [SO]: Python Ctypes - loading dll throws OSError: [WinError 193] %1 is not a valid Win32 application (@CristiFati's answer)

• [SO]: How to check for DLL dependency?

• [SO]: C DLL loads in C++ program, not in python Ctypes (@MarkTolonen's answer)

Duct

Thanks, great answer! I was aware of the dependency on %PATH%, forgot to mention it. Dependencies.exe seems to be the best suggestion here, thanks again. – Hanhhank

This didn't even relate to my problem, but I upvoted just by how well it is answered haha – Declarative

@WilliamBrochensquejunior: Thank you, can you share your problem, maybe this can be improved to answer it as well, or maybe there are other answers who address it (considering that your problem is the same or similar to this one). Note that it illustrates the process of investigating this kind of issues, with a specific example. – Duct

I think I can the solution on similar questions, so not that much of a problem! Thank you anyway. I will create a question about it if nothing works! – Declarative