Login/Register
Can't import dll module in Python
Asked Answered
pythonwindowswinapictypesuvc
I

6

21

I've been stressin for a few days trying to compile a modified version of libuvc on windows and now that I've finally done it, I can't seem to load it on Python. This lib that I've already compiled and successfully imported using the same version of Python on Linux machines, doesn't like w10 at all.

System

  • win 10 64 bit
  • python 3.8 64 bit
  • libusb 1.022
  • libuvc.dll compiled with MinGW64

Problem

When trying the 

import ctypes
import ctypes.util
name = ctypes.util.find_library('libuvc')
lib = ctypes.cdll.LoadLibrary(name)

I get the following error:

Could not find module 'C:\Program Files (x86)\libuvc\lib\libuvc.dll'.
Try using the full path with constructor syntax. 
Error: could not find libuvc!

The issue is that the file exists since it was found by util.find_library, but python doesn't think it is where it is, or maybe the output is just the default. What am I missing here? What could be failing to be unable to not just load the module, but find it? I'm sorry I don't have more output than this.

P.S: I've tried reformatting the string in different ways, but the message doesn't change.

Intact answered 13/12, 2019 at 23:6 Comment(6)
Perhaps something to do with permissions on that file? Give process monitor a try and see what happened while accessing that file. – Nicollenicolson
what would be an error message that would indicate permission error? cause there is a lot of feed in process monitor. What am i supposed to be looking for? there is a lot of successful readings of the dll. – Intact
Filter by the path you're trying to access. Or try to count the Result column and look for some permission denied or similar – Nicollenicolson
See this answer for how to use Process Monitor to find a DLL issue. It may be a dependent DLL in a different path it can't find, so limiting by path could miss something. Limit by process and CreateFile and look at the end of the list after trying to load your DLL. – Colchis
I've taken the liberty of filing a minor bug with python to resolve the discrepancy between the docs and the actual behavior: bugs.python.org/issue42114 – Shorter
Can someone tell me what Python means by "constructor syntax"? – Agog
H
26

Starting with Python 3.8, the .dll search mechanism has changed (Win specific).

According to [Python.Docs]: os.add_dll_directory(path) (emphasis is mine):

Add a path to the DLL search path.

This search path is used when resolving dependencies for imported extension modules (the module itself is resolved through sys.path), and also by ctypes.

...

Availability: Windows.

So, you could do:

os.add_dll_directory("${path_to_working_dlls_directoy}")

where ${path_to_working_dlls_directoy} is a placeholder for the actual path and it (obviously) should be replaced by it.

You can check [SO]: PyWin32 and Python 3.8.0 (@CristiFati's answer) (which although it seems very different, has the same cause), for more details.

P.S.: Nix OSes are not impacted.


Update #0

While I based my answer on (official) documentation, [SO]: Can't import dll module in Python (@MadPhysicist's answer) (great answer BTW) came with the why part.
However, there are some aspects that need to be clarified.

Everything I stated about os.add_dll_directory is correct and still applies. According to [MS.Learn]: LoadLibraryExA function (libloaderapi.h) (emphasis is mine):

  • LOAD_LIBRARY_SEARCH_DEFAULT_DIRS
    0x00001000

    This value is a combination of LOAD_LIBRARY_SEARCH_APPLICATION_DIR, LOAD_LIBRARY_SEARCH_SYSTEM32, and LOAD_LIBRARY_SEARCH_USER_DIRS. Directories in the standard search path are not searched. This value cannot be combined with LOAD_WITH_ALTERED_SEARCH_PATH.
    This value represents the recommended maximum number of directories an application should include in its DLL search path

  • LOAD_LIBRARY_SEARCH_USER_DIRS
    0x00000400

    If this value is used, directories added using the AddDllDirectory or the SetDllDirectory function are searched for the DLL and its dependencies.

This is the mechanism used by default (winmode=None, as LOAD_LIBRARY_SEARCH_DEFAULT_DIRS is passed to LoadLibraryEx).
When setting winmode=0, Win default search mechanism kicks in. [MS.Learn]: Dynamic-Link Library Search Order - Standard Search Order for Desktop Applications states (last item - regardless of SafeDllSearchMode, emphasis still mine):

  1. The directories that are listed in the PATH environment variable. Note that this does not include the per-application path specified by the App Paths registry key. The App Paths key is not used when computing the DLL search path.

I prepared a small example.
Python code attempts to load (via CTypes) dll00.dll located in the same dir, which in turn is linked to (depends on) another one (dll01.dll) located in a different dir (subdir00).

  • dll00.c:

    #include <stdio.h>

#if defined(_WIN32)
#  define DLL00_EXPORT_API __declspec(dllexport)
#else
#  define DLL00_EXPORT_API
#endif


int dll01Func00();

#if defined(__cplusplus)
extern "C" {
#endif

DLL00_EXPORT_API int dll00Func00();

#if defined(__cplusplus)
}
#endif


int dll00Func00() {
    printf("%s - %d - %s\n", __FILE__, __LINE__, __FUNCTION__);
    dll01Func00();
    return 0;
}

  • dll01.c:

    #include <stdio.h>

#if defined(_WIN32)
#  define DLL01_EXPORT_API __declspec(dllexport)
#else
#  define DLL01_EXPORT_API
#endif


#if defined(__cplusplus)
extern "C" {
#endif

DLL01_EXPORT_API int dll01Func00();

#if defined(__cplusplus)
}
#endif


int dll01Func00() {
    printf("%s - %d - %s\n", __FILE__, __LINE__, __FUNCTION__);
    return 0;
}

  • code00.py:

    #!/usr/bin/env python

import argparse
import ctypes as cts
import os
import sys


DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")

METH_ADDLLDIR = "a"
METH_PATH = "p"
METHS = (
    METH_ADDLLDIR,
    METH_PATH,
)


def parse_args(*argv):
    parser = argparse.ArgumentParser(description="Python .dll search path (Win) example")
    parser.add_argument("--path", "-p", choices=METHS)
    parser.add_argument("--winmode", "-w", type=int)

    args, unk = parser.parse_known_args()
    if unk:
        print("Warning: Ignoring unknown arguments: {:}".format(unk))
    return args.path, args.winmode


def main(*argv):
    meth, wmod = parse_args()
    print("PATH (original): {:}\n".format(os.environ.get("PATH")))
    print("Using winmode={:}".format(wmod))
    if meth is not None:
        subdir = os.path.join(os.path.abspath(os.path.dirname(__file__)), "subdir00")
        if meth == METH_ADDLLDIR:
            add_dll_directory = getattr(os, "add_dll_directory", None)
            if add_dll_directory:
                os.add_dll_directory(subdir)
                print("Using AddDllDirectory()\n")
        elif meth == METH_PATH:
            os.environ["PATH"] = os.pathsep.join((os.environ.get("PATH", ""), subdir))
            print("Using %PATH%\n")
    dll00 = cts.CDLL(DLL_NAME, winmode=wmod)
    print("Dll: {:}".format(dll00))
    if False:  # No need to actually call the function
        dll00Func00 = dll00.dll00Func00
        dll00Func00.argtypes = ()
        dll00Func00.restype = cts.c_int
        res = dll00Func00()
        print("\n{0:s} returned: {1:d}".format(dll00Func00.__name__, res))


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.\n")
    sys.exit(rc)

Notes:

  • There are 2 variables here (alterable via command line arguments):

    • path - how subdir00 should be added to .dll search paths:

      1. Do nothing (don't add it)

      2. Pass it to os.add_dll_directory

      3. Append it to %PATH%

    • winmode - to be passed to CDLL initializer (only illustrating for 0 and None)

    As a consequence, 6 combinations emerge

Output:

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q059330863]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Auxiliary\Build\vcvarsall.bat" x64 > nul

[prompt]> tree /a /f
Folder PATH listing for volume SSD0-WORK
Volume serial number is AE9E-72AC
E:.
|   code00.py
|   dll00.c
|   dll01.c
|   test.bat
|
\---subdir00

[prompt]> :: Build .dlls
[prompt]> cl /nologo /DDLL /MD dll01.c  /link /NOLOGO /DLL /OUT:subdir00\dll01.dll
dll01.c
   Creating library subdir00\dll01.lib and object subdir00\dll01.exp

[prompt]>
[prompt]> cl /nologo /DDLL /MD dll00.c  /link /NOLOGO /DLL /OUT:dll00.dll subdir00\dll01.lib
dll00.c
   Creating library dll00.lib and object dll00.exp

[prompt]>
[prompt]> tree /a /f
Folder PATH listing for volume SSD0-WORK
Volume serial number is AE9E-72AC
E:.
|   code00.py
|   dll00.c
|   dll00.dll
|   dll00.exp
|   dll00.lib
|   dll00.obj
|   dll01.c
|   dll01.obj
|   test.bat
|
\---subdir00
        dll01.dll
        dll01.exp
        dll01.lib


[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" code00.py -h
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

usage: code00.py [-h] [--path {a,p}] [--winmode WINMODE]

Python .dll search path (Win) example

options:
  -h, --help            show this help message and exit
  --path {a,p}, -p {a,p}
  --winmode WINMODE, -w WINMODE

[prompt]>
[prompt]> :: Going through combinations. When an argument is not passed, its default value is None
[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" code00.py
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

PATH (original): c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\\Extensions\Microsoft\IntelliCode\CLI;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\VCPackages;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TestWindow;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\MSBuild\Current\bin\Roslyn;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\\x64;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4
.8 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\devinit;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\\MSBuild\Current\Bin;C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Install\pc064\Docker\Docker\Version\Docker\resources\bin;C:\ProgramData\DockerDesktop\version-bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\dotnet\;C:\Users\cfati\AppData\Local\Programs\Python\Launcher\;e:\Work\Dev\Utils\current\Win;e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts;C:\Us
ers\cfati\AppData\Local\Microsoft\WindowsApps;C:\Users\cfati\.dotnet\tools;;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\Linux\bin\ConnectionManagerExe

Using winmode=None
Traceback (most recent call last):
  File "e:\Work\Dev\StackOverflow\q059330863\code00.py", line 57, in <module>
    rc = main(*sys.argv[1:])
  File "e:\Work\Dev\StackOverflow\q059330863\code00.py", line 44, in main
    dll00 = cts.CDLL(DLL_NAME, winmode=wmod)
  File "c:\Install\pc064\Python\Python\03.10\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'e:\Work\Dev\StackOverflow\q059330863\dll00.dll' (or one of its dependencies). Try using the full path with constructor syntax.

[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" code00.py -w 0
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

PATH (original): c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\\Extensions\Microsoft\IntelliCode\CLI;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\VCPackages;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TestWindow;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\MSBuild\Current\bin\Roslyn;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\\x64;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4
.8 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\devinit;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\\MSBuild\Current\Bin;C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Install\pc064\Docker\Docker\Version\Docker\resources\bin;C:\ProgramData\DockerDesktop\version-bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\dotnet\;C:\Users\cfati\AppData\Local\Programs\Python\Launcher\;e:\Work\Dev\Utils\current\Win;e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts;C:\Us
ers\cfati\AppData\Local\Microsoft\WindowsApps;C:\Users\cfati\.dotnet\tools;;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\Linux\bin\ConnectionManagerExe

Using winmode=0
Traceback (most recent call last):
  File "e:\Work\Dev\StackOverflow\q059330863\code00.py", line 57, in <module>
    rc = main(*sys.argv[1:])
  File "e:\Work\Dev\StackOverflow\q059330863\code00.py", line 44, in main
    dll00 = cts.CDLL(DLL_NAME, winmode=wmod)
  File "c:\Install\pc064\Python\Python\03.10\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module './dll00.dll' (or one of its dependencies). Try using the full path with constructor syntax.

[prompt]>
[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" code00.py -p a
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

PATH (original): c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\\Extensions\Microsoft\IntelliCode\CLI;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\VCPackages;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TestWindow;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\MSBuild\Current\bin\Roslyn;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\\x64;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4
.8 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\devinit;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\\MSBuild\Current\Bin;C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Install\pc064\Docker\Docker\Version\Docker\resources\bin;C:\ProgramData\DockerDesktop\version-bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\dotnet\;C:\Users\cfati\AppData\Local\Programs\Python\Launcher\;e:\Work\Dev\Utils\current\Win;e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts;C:\Us
ers\cfati\AppData\Local\Microsoft\WindowsApps;C:\Users\cfati\.dotnet\tools;;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\Linux\bin\ConnectionManagerExe

Using winmode=None
Using AddDllDirectory()

Dll: <CDLL 'e:\Work\Dev\StackOverflow\q059330863\dll00.dll', handle 7ffe6aaf0000 at 0x1f896d9ffd0>

Done.


[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" code00.py -p a -w 0
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

PATH (original): c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\\Extensions\Microsoft\IntelliCode\CLI;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\VCPackages;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TestWindow;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\MSBuild\Current\bin\Roslyn;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\\x64;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4
.8 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\devinit;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\\MSBuild\Current\Bin;C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Install\pc064\Docker\Docker\Version\Docker\resources\bin;C:\ProgramData\DockerDesktop\version-bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\dotnet\;C:\Users\cfati\AppData\Local\Programs\Python\Launcher\;e:\Work\Dev\Utils\current\Win;e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts;C:\Us
ers\cfati\AppData\Local\Microsoft\WindowsApps;C:\Users\cfati\.dotnet\tools;;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\Linux\bin\ConnectionManagerExe

Using winmode=0
Using AddDllDirectory()

Traceback (most recent call last):
  File "e:\Work\Dev\StackOverflow\q059330863\code00.py", line 57, in <module>
    rc = main(*sys.argv[1:])
  File "e:\Work\Dev\StackOverflow\q059330863\code00.py", line 44, in main
    dll00 = cts.CDLL(DLL_NAME, winmode=wmod)
  File "c:\Install\pc064\Python\Python\03.10\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module './dll00.dll' (or one of its dependencies). Try using the full path with constructor syntax.

[prompt]>
[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" code00.py -p p
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

PATH (original): c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\\Extensions\Microsoft\IntelliCode\CLI;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\VCPackages;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TestWindow;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\MSBuild\Current\bin\Roslyn;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\\x64;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4
.8 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\devinit;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\\MSBuild\Current\Bin;C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Install\pc064\Docker\Docker\Version\Docker\resources\bin;C:\ProgramData\DockerDesktop\version-bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\dotnet\;C:\Users\cfati\AppData\Local\Programs\Python\Launcher\;e:\Work\Dev\Utils\current\Win;e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts;C:\Us
ers\cfati\AppData\Local\Microsoft\WindowsApps;C:\Users\cfati\.dotnet\tools;;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\Linux\bin\ConnectionManagerExe

Using winmode=None
Using %PATH%

Traceback (most recent call last):
  File "e:\Work\Dev\StackOverflow\q059330863\code00.py", line 57, in <module>
    rc = main(*sys.argv[1:])
  File "e:\Work\Dev\StackOverflow\q059330863\code00.py", line 44, in main
    dll00 = cts.CDLL(DLL_NAME, winmode=wmod)
  File "c:\Install\pc064\Python\Python\03.10\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'e:\Work\Dev\StackOverflow\q059330863\dll00.dll' (or one of its dependencies). Try using the full path with constructor syntax.

[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" code00.py -p p -w 0
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

PATH (original): c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\\Extensions\Microsoft\IntelliCode\CLI;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\VCPackages;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TestWindow;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\MSBuild\Current\bin\Roslyn;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Team Tools\Performance Tools;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\\x64;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4
.8 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\devinit;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\\MSBuild\Current\Bin;C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\Tools\;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Install\pc064\Docker\Docker\Version\Docker\resources\bin;C:\ProgramData\DockerDesktop\version-bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\dotnet\;C:\Users\cfati\AppData\Local\Programs\Python\Launcher\;e:\Work\Dev\Utils\current\Win;e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts;C:\Us
ers\cfati\AppData\Local\Microsoft\WindowsApps;C:\Users\cfati\.dotnet\tools;;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\Common7\IDE\VC\Linux\bin\ConnectionManagerExe

Using winmode=0
Using %PATH%

Dll: <CDLL './dll00.dll', handle 7ffe6aaf0000 at 0x142a7a73cd0>

Done.

Only 2 combinations (as expected) were successful (use subdir00):

  • Pass it to os.add_dll_directory (with winmode None)

  • Add it to %PATH% (with winmode 0)

Conclusions

There are more ways of adding dirs where to search for (dependent) .dlls:

  1. os.add_dll_directory:

    • Default (and recommended) option (there are 3rd-party modules that load .dlls this way)

    • Consistent with the way Python loads extension module (.pyd) dependencies

  2. %PATH% (and alter winmode):

    • Works with older (< v3.8) Python versions

    • Seems closer with Nix - where paths are typically added to ${LD_LIBRARY_PATH} (DYLD_LIBRARY_PATH, LIBPATH, SHLIB_PATH, ...)

    • Can be customized by winmode's value

  3. Change directory to .dll location (more like a workaround):

    • Won't work if .dlls are in multiple directories (can't have multiple CWDs at the same time)

Related (more or less):

Hyposthenia answered 14/12, 2019 at 9:3 Comment(5)
I've expanded to show what exactly changed in the search mechanism. – Shorter
To remind myself every time I go here, this does not work on unix or macOS. add_dll_directory is only found on windows. – Salsify
@Konchog: that's true, the question is tagged Windows, it's about .dlls, the doc also mentions it. Anyway modified the answer to be more explicit. For other OSes, it goes the "regular" way: e.g. setting LD\LIBRARY_PATH. #58646468 – Hyposthenia
On MacOS, The following seem to have an effect in os.environment 'DYLD_FRAMEWORK_PATH' 'DYLD_LIBRARY_PATH' 'DYLD_FALLBACK_FRAMEWORK_PATH' 'DYLD_FALLBACK_LIBRARY_PATH' – Salsify
Nope, only winmode 1 worked for me(even without using os.add_dll_directory. ), but it fails with segmentation error when accessing the function. winmode 2 compiles but does not help as it fails to recognize the function where I am setting some args. I am using cygwin on windows 11. Python is at 3.10 – Cuprite
S
26

A year late, but I've figured out what is going on and how to fix it. If you look at the code for ctypes.CDLL on around line 340, you can see that the docs are actually incorrect. The code defines the constructor as

def __init__(self, name, mode=DEFAULT_MODE, handle=None,
             use_errno=False, use_last_error=False, winmode=None):

The docs, however, say winmode=0. If you look at line 358, you can see that it matters quite a bit. When winmode=None, the search mode used by _ctypes.LoadLibrary in line 374 (aliased as _dlopen in line 110) is set to nt._LOAD_LIBRARY_SEARCH_DEFAULT_DIRS on line 363. This search mode does not appear to respond to changes to os.environ['PATH'], sys.path or os.add_dll_directory.

However, if you bypass that setting by using winmode=0 instead of None, the library appears to load fine. Zero is a valid mode for a full path (as would be nt._LOAD_WITH_ALTERED_SEARCH_PATH). A full list of modes is available here: https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa, under the dwFlags parameter.

As @CristiFati indicates, the behavior changed in Python 3.8. This happened because prior to that, the winmode parameter did not exist. Instead, mode was used directly, with a default value of ctypes.DEFAULT_MODE, which happens to correspond to a zero and work on all platforms.

Python bug report to resolve discrepancy: https://bugs.python.org/issue42114

Shorter answered 21/10, 2020 at 21:39 Comment(15)
Yes, I remember that I went through the CTypes code a year ago. You are right, the doc does not reflect what's in the code (hmm, I wonder how come I missed that). However, you're not correct about os.add_dll_directory. With winmode=None things work fine because of LOAD_LIBRARY_SEARCH_USER_DIRS which is sensitive to AddDllDirectory. – Hyposthenia
With LOAD_LIBRARY_SEARCH_DEFAULT_DIRS, the current directory and PATH are ignored for the sake of security, but the default search path includes the application directory (of python.exe) and System32 directory, and any directory added via os.add_dll_directory. If all dependent DLLs are distributed in the same directory, then nothing special should be needed other than using a qualified path to ensure that it uses the flag LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR. Otherwise, for a secure load, as used by default, you have to manually include every required directory via os.add_dll_directory. – Blanton
@CristiFati. It didn't work on win 10. I would have expected the flags to work as you described. Perhaps there's something different in the security that Eryk is referring to. – Shorter
@MadPhysicist: I just created a small example (that I can add to my answer if you like): Python code that loads a .dll which depends on another .dll (located in a different dir). As expected, it doesn't work OOTB. In Python 3.7 adding that dir to %PATH% fixes it, but not on Python 3.8. Adding that same dir with os.add_dll_directory solves the problem in Python 3.8. I am on Win 10. Maybe on older OS versions it didn't work because the flag does not exist without installing KB2533623? – Hyposthenia
You have to analyze the dependent DLLs if you want a secure load. Try it first with winmode=0. Then in Process Explorer, switch the view in the bottom pane to show the full path of loaded DLLs. All directories except the application directory (of python.exe), the System32 directory, and the directory that contains the DLL that was directly loaded via ctypes.CDLL have to be added manually via os.add_dll_directory before calling ctypes.CDLL. If dependencies are in the DLL directory, make sure the path passed to CDLL is qualified in order to include its directory in the search. – Blanton
@ErykSun: I was talking about LOAD_LIBRARY_SEARCH_USER_DIRS. – Hyposthenia
@ErykSun: I was trying to imagine a scenario that os.add_dll_directory wouldn't do the trick (as MadPhysicist suggests in the 3rd comment). Probably if running Python 3.8 on an older Win version without that KB, the flag would not be present there (and most likely the functionality (search AddDllDirectory paths) associated with it would not work either). – Hyposthenia
@ErykSun. I think I see your point about dependency analysis. I'll check tomorrow. – Shorter
@ErykSun. Thanks for the tip. I carefully added all the dependent folders and everything started working as expected. – Shorter
Please change "does not appear to respond to changes to os.environ['PATH'], sys.path or os.add_dll_directory" to an accurate statement such as "does not include os.environ['PATH'] and the current working directory". sys.path is for Python modules; there's no need to mention it here. Also, note that mode doesn't matter in Windows; it used to be passed to but not used by low-level _ctypes.LoadLibrary; hence the need for winmode. It would also be good to summarize some steps needed to support the secure DLL search path with the default winmode=None and os.add_dll_directory. – Blanton
@ErykSun. I'll read through the comments you made, as well as some other stuff tonight and rewrite this answer. – Shorter
@bomben. Comments are not a good place to ask a question that's clearly going to take a lot of detail to answer. Please ask a separate question with a minimal reproducible example – Shorter
@bomben. Possibly, but you are not providing nearly enough information in the comments, and please don't try – Shorter
@MadPhysicist #70937583 – Fantail
I wish I could upvote this twice, thanks much for this answer! – Ikey
V
4

I partially agree with @MadPhysicist's answer, but I have Python 3.9, not 3.8, and with winmode=0 error haven't disappeared. But with winmode=1 everything is working!

Valletta answered 11/8, 2022 at 1:51 Comment(1)
https://mcmap.net/q/21425/-calling-a-c-function-with-a-char-argument-through-python-ctypes-difference-python2-and-python3/4788546. – Hyposthenia
I
2

OK so i fixed it, it was required that i changed the working directory to where the script was being executed before loading the dll from the same place.

os.chdir('path_to_working_dlls_directoy')

not entirely sure why this helped though.

Intact answered 14/12, 2019 at 2:18 Comment(1)
Note that this is just a workaround. If there are multiple folders with .dlls, it won't work. Try the suggestion from my answer. -1. – Hyposthenia
K
-2

Just "Visual C++ Redistributable Package per Visual Studio 2013". The problem will be solved.

Karyn answered 15/11, 2022 at 6:34 Comment(1)
This is incorrect. -1. – Hyposthenia
C
-3

You can specify the path to the library

import snap7
import struct
from snap7.common import Snap7Library
from snap7.util import *

# If you are using a different location for the library
Snap7Library(lib_location='C:/snap7/snap7.dll')
load_library() #Testing library is correctly <WinDLL 'C:\snap7\snap7.dll', handle 7ff9d5d90000 at 0x1a5a0417640>

plc = snap7.client.Client()
plc.connect("10.112.115.10",0,1)
Chong answered 16/9, 2020 at 3:42 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.