Why does PyImport_Import fail to load a module from the current directory?

Question

I'm trying to run the embedding example and I can't load a module from the current working directory unless I explicitly add it to sys.path then it works:

PyRun_SimpleString("import sys");
PyRun_SimpleString("sys.path.append(\".\")"); 

Shouldn't Python look for modules in the current directory ?

Edit1: Tried just importing the module with:

Py_Initialize();
PyRun_SimpleString("import multiply"); 

And it still fails with the following error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named multiply

Edit2: From the sys.path docs:

If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first.

Not sure what it means by not available, but if I print sys.path[0] it's not empty:

/usr/lib/pymodules/python2.7

Radioelement

python should, have you tried printing out os.getcwd() to check you are where you think you are? – Choking

try from __main__ import multiply assuming that multiply a function defined in a file as per the tutorial – Choking

Answer 1

You need to call PySys_SetArgv(int argc, char **argv, int updatepath) for the relative imports to work. This will add the path of the script being executed to sys.path if updatepath is 0 (refer to the docs for more information).

The following should do the trick

#include <Python.h>

int
main(int argc, char *argv[])
{
  Py_SetProgramName(argv[0]);  /* optional but recommended */
  Py_Initialize();
  PySys_SetArgv(argc, argv); // must call this to get sys.argv and relative imports
  PyRun_SimpleString("import os, sys\n"
                     "print sys.argv, \"\\n\".join(sys.path)\n"
                     "print os.getcwd()\n"
                     "import thing\n" // import a relative module
                     "thing.printer()\n");
  Py_Finalize();
  return 0;
}

Choking

I think you should include the following sentence from the docs and modify your sample accordingly: These parameters are similar to those passed to the program’s main() function with the difference that the first entry should refer to the script file to be executed rather than the executable hosting the Python interpreter. – Deloris

Answer 2

Although other answers offer great answers as how to include the current directory, I think the actual answer to the original question may be in this:

Automatically adding current directory to sys.path is the "safepath" concept. But look:

int safepath
...
Default: 0 in Python config, 1 in isolated config.

https://docs.python.org/3/c-api/init_config.html#c.PyConfig.safe_path

The Isolated Configuration can be used to embed Python into an application. It isolates Python from the system. For example, environment variables are ignored, the LC_CTYPE locale is left unchanged and no signal handler is registered.

https://docs.python.org/3/c-api/init_config.html#init-config

So as it seems, embedded mode is run as isolated mode, which specifically disables this behavior by default.

This answer is for Python 3, where this behavior is still observable.

Wedlock

Answer 3

What I had to do with python 3.5 is PySys_SetPath to be able to import from the cwd location:

QString qs = QDir::currentPath();
std::wstring ws = qs.toStdWString();
PySys_SetPath(ws.data());

The Qs in it is Qt.

Importune

Answer 4

To load a module from the current directory, there are many options.

1. You need to configure Python (including the path configuration) through an initialization struct. See Python Initialization Configuration.

Py_Initialize():
//...
PyConfig config;
PyConfig_InitPythonConfig(&config);
config.module_search_paths_set = 1;
PyWideStringList_Append(&config.module_search_paths, L"."); // append local dir to path
Py_InitializeFromConfig(&config);
//...
pModule = PyImport_Import(pName); // import module

2. A one-liner option is to run PySys_SetArgv(0, NULL); also after Py_Initialize() to prepend current directory to Python's Path. Although this option is deprecated since Python 3.11, it is simple and a valid option in many situations. This option takes advantage of the behavior of PySys_SetArgvEx(int argc, char **argv, int updatepath) when satisfying:

• non-zero updatepath
• argc = 0 or argv not pointing to an existing filename.

See docs. Valid from Python 2.7 onwards. For adding a specific directory, argv[0] should contain the path to a Python script in that directory (for example "NULL").PySys_SetArgv sets the updatepath value correctly for you. Although, as mentioned, this option is currently deprecated.

---

Example Project

Load a module defined in a Python script located in the current directory and call a function from that module, all specified with input arguments to a C application. Checks deleted for simplicity. This example is closely related to the example provided in the documentation page for embedding Python in a C/C++ application.

./hello.py

def sayHello1():
  print("Hello from sayHello1!")
  return 1

def sayHello2():
  print("Hello from sayHello2!")
  return 2

./main.c

(checks removed for simplicity)

#define PY_SSIZE_T_CLEAN
#include <Python.h>

int main(int argc, char *argv[])
{
  PyObject *pName = NULL;
  PyObject *pModule = NULL;
  PyObject *pFunc = NULL;
  PyObject *pValue = NULL;

  Py_Initialize();
// PySys_SetArgv(0, NULL); // Option 2 [deprecated]

// Option 1 - Python Initialization api
  PyConfig config;
  PyConfig_InitPythonConfig(&config);
  config.module_search_paths_set = 1;
  PyWideStringList_Append(&config.module_search_paths, L".");
  Py_InitializeFromConfig(&config);
// ...

  pName = PyUnicode_FromString(argv[1]);
  pModule = PyImport_Import(pName);
  Py_XDECREF(pName);

  pFunc = PyObject_GetAttrString(pModule, argv[2]);
  pValue = PyObject_CallNoArgs(pFunc); // actual call to the fcn

  Py_XDECREF(pValue);
  Py_XDECREF(pFunc);
  Py_XDECREF(pModule);
  
  Py_Finalize();
  return 0;
}

Compilation (cmake):

cmake_minimum_required(VERSION 3.2)
project(python_c_api)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

find_package(Python COMPONENTS Interpreter Development)
cmake_path(GET Python_EXECUTABLE PARENT_PATH Python_BIN_DIR) 

set(config_cmd_flags "--cflags")
execute_process(COMMAND ${Python_BIN_DIR}/python${Python_VERSION_MAJOR}.${Python_VERSION_MINOR}-config ${config_cmd_flags} 
  OUTPUT_VARIABLE Python_FLAGS)

add_executable(python_extension main.c)
target_link_libraries(python_extension PUBLIC python${Python_VERSION_MAJOR}.${Python_VERSION_MINOR})
target_link_directories(python_extension PUBLIC ${Python_LIBRARY_DIRS})
target_include_directories(python_extension PUBLIC ${Python_INCLUDE_DIRS})
set_target_properties(python_extension PROPERTIES 
  OUTPUT_NAME "call"
  RUNTIME_OUTPUT_DIRECTORY ${PROJECT_SOURCE_DIR}
  )
separate_arguments(Python_FLAGS_NORM UNIX_COMMAND "${Python_FLAGS}")
target_compile_options(python_extension PUBLIC ${Python_FLAGS_NORM})

cmake .
cmake --build .

Execution: (checks deleted for simplicity. Call the app as shown here).

./call hello sayHello1
Hello from sayHello1!

./call hello sayHello2
Hello from sayHello2!

Edit: This solution appends the current working directory to the search path, not the directory where the binary executable containing the embedded python is stored. Thus, if the application is called from a different directory, that directory will be appended to the search path. Adding the path of the binary executable app (containing the python interpreter embedded) to the search path is not trivial and there doesn't seem to exist an easy cross-platform solution. If this is required, I recommend wrap the app in another app or script running in the correct directory.

Karlynkarma

Answer 5

I had exactly the same problem and I solved it just by adding the Py_Initialize(); and Py_Finalize();

Hope that can help you

Blackshear