PyImport_ImportModule, possible to load module from memory?

Question

I embedded python in my C++ program.

I use PyImport_ImportModule to load my module written in a .py file. But how can I load it from memory? Let's say my .py file is encrypted, so I need to first decrypt it and feed the code to python to execute.

Moreover, it'd be nice if I could bypass/intercept or modify the import mechanism, so that doesn't load modules from the filesystem but my own memory blocks, how/can I do that?

Suffragan

Answer 1

The following example shows how to define a module from a C string:

#include <stdio.h>
#include <Python.h>
int main(int argc, char *argv[])
{
    Py_Initialize();
    PyRun_SimpleString("print('hello from python')");

    // fake module
    char *source = "__version__ = '2.0'";
    char *filename = "test_module.py";

    // perform module load
    PyObject *builtins = PyEval_GetBuiltins();
    PyObject *compile = PyDict_GetItemString(builtins, "compile");
    PyObject *code = PyObject_CallFunction(compile, "sss", source, filename, "exec");
    PyObject *module = PyImport_ExecCodeModule("test_module", code);

    PyRun_SimpleString("import test_module; print(test_module.__version__)");

    Py_Finalize();
    return 0;
}

output:

hello from python
version: 2.0

You can read about import hooks in the docs. You will need to define a class with find_module and load_module methods. Something like the following should work:

PyObject* find_module(PyObject* self, PyObject* args) {
    // ... lookup args in available special modules ...
    return Py_BuildValue("B", found);
}

PyObject* load_module(PyObject* self, PyObject* args) {
    // ... convert args into filname, source ...
    PyObject *builtins = PyEval_GetBuiltins();
    PyObject *compile = PyDict_GetItemString(builtins, "compile");
    PyObject *code = PyObject_CallFunction(compile, "sss", source, filename, "exec");
    PyObject *module = PyImport_ExecCodeModule("test_module", code);
    return Py_BuildValue("O", module);
}

static struct PyMethodDef methods[] = {
    { "find_module", find_module, METH_VARARGS, "Returns module_loader if this is an encrypted module"},
    { "load_module", load_module, METH_VARARGS, "Load an encrypted module" },
    { NULL, NULL, 0, NULL }
};

static struct PyModuleDef modDef = {
    PyModuleDef_HEAD_INIT, "embedded", NULL, -1, methods, 
    NULL, NULL, NULL, NULL
};

static PyObject* PyInit_embedded(void)
{
    return PyModule_Create(&modDef);
}

int main() {
    ...
    PyImport_AppendInittab("embedded", &PyInit_embedded);
    PyRun_SimpleString("\
import embedded, sys\n\
class Importer:\n\
    def find_module(self, fullpath):\n\
        return self if embedded.find_module(fullpath) else None\n\
    def load_module(self, fullpath):\n\
        return embedded.load_module(fullpath)\n\
sys.path_hooks.insert(0, Importer())\n\
");
    ...
}

Neurologist