How to import python file from git submodule

Question

I've a project which uses git submodules. In my python file I want to use functions from another python file in the submodule project.

In order to work I had to add the init.py file to all subfolders in the path. My folder tree is the following:

myproj
├── gitmodules
│   ├── __init__.py
│   ├── __init__.pyc
│   └── mygitsubmodule
│       ├── __init__.py
│       ├── __init__.pyc
│       └── file.py
└── myfile.py

Is there any way to make it work without touching mygitsubmodule ?

Thanks

Fivepenny

Answer 1

you can add to sys.path in the file you want to be able to access the module, something like:

import sys
sys.path.append("/home/me/myproj/gitmodules")
import mygitsubmodule

This example is adding a path as a raw string to make it clear what's happening. You should really use the more sophisticated, system independent methods described below to determine and assemble the path.

Also, I have found it better, when I used this method, to use sys.path.insert(1, .. as some functionality seems to rely of sys.path[0] being the starting directory of the program.

Vellum

I was hopping a less trickier way (editing the sys.path just for an import..), but I guess that it may not be possible to do it in another way :) – Fivepenny

@pedrorijo91: sys.path is the Python load path, not to be confused with the $PATH environment variable or it's equivalents. It's quite lightweight to add to, and doesn't alter your environment. Edit: I would suggest working out the path at runtime instead of hard-coding it. Something involving the current path (os.path.dirname(os.path.abspath(__file__))) and os.path.join – Nickelous

See https://mcmap.net/q/427858/-git-submodule-import-statements-failing for how to do this for the entire package at once. – Bessette

Answer 2

I am used to avoiding modifying sys.path.

The problem is, when using git submodule, submodule is a project directory, not a Python package. There is a "gap" between your module and that package, so you can't import.

Suppose you have created a submodule named foo_project, and there is a foo package inside.

.
├── foo_project
│   ├── README.rst
│   └── foo
│       └── __init__.py
└── main.py

My solution will be creating a soft link to expose that package to your module:

ln -s foo_project/foo foo

.
├── foo_project
│   ├── README.rst
│   └── foo
│       └── __init__.py
├── foo -> foo_project/foo
└── main.py

Now you can import foo in the main.py.

Burning

That might work, if the imported file doesn't depend on other files within foo_project. It unfortunately fails, if there are json files loaded or other stuff. – Nananne

Answer 3

For reference,

       from submodulefolder.file import func_name   

or

       import submodulefolder.file as lib_name

where file excludes the extension of file.py, seems to work in relative terms without modifying the subfolder / git submodule with a init.py since python 3.3+, as shown here.

Tested on py3.8.5 linux native and py3.7.8 anaconda Windows, both in Spyder's Ipython-console, as well as natively on linux via terminal.

Pyrophoric