Import a file from a subdirectory?

Question

I have a file called tester.py, located on /project.

/project has a subdirectory called lib, with a file called BoxTime.py:

/project/tester.py
/project/lib/BoxTime.py

I want to import BoxTime from tester. I have tried this:

import lib.BoxTime

Which resulted:

Traceback (most recent call last):
  File "./tester.py", line 3, in <module>
    import lib.BoxTime
ImportError: No module named lib.BoxTime

Any ideas how to import BoxTime from the subdirectory?

EDIT

The __init__.py was the problem, but don't forget to refer to BoxTime as lib.BoxTime, or use:

import lib.BoxTime as BT
...
BT.bt_function()

Coelacanth

Seems he is importing lib/BoxTime to tester. – Assurance

Python 3.3+ has Implicit Namespace Packages that allow it to create a packages without an __init__.py file. Check Is __init__.py not required for packages in Python 3.3+. – Belanger

Answer 1

Take a look at the Packages documentation (Section 6.4).

In short, you need to put a blank file named

__init__.py

in the lib directory.

Heyduck

Why does it feel hacky? It's the way python marks safe/available import directories. – Teapot

Not only it marks safe/available import directories, but also provides a way to run some initialization code when importing a directory name. – Trafficator

Yes this is hacky and even dirty, and in my opinion the language shouldn't impose its way of loading files across the filesystem. In PHP we solved the problem by letting the userland code register multiple autoloading functions that are called when a namespace/class is missing. Then the community has produced the PSR-4 standard and Composer implements it, and nowadays nobody has to worry about that. And no stupid hardcoded __init__ files (but if you want it, just register an autoloading hook ! This is the difference between hacky and hackable). – Kurtzman

@YaShChaudhary It didn't work for me until I changed the reference to library: foldername.modulename – Ellsworth

@Ellsworth Your foldername.modulename worked for me doing an explict dynamic load with importlib.import_module. – Fox

The necessary __init__.py file was at least not comfortable, may it be hacky or not. And this point got resolved in Python 3.3: A package can be made up without the file. – Frolicsome

See also stackoverflow.com/questions/37139786/… – Oolite

Related, but not OP's wish, is to install the lib into the local pip: cd your-git-work-tree && pip install -e . Then you can simply use the library via the pip system. – Ringe

Answer 2

• Create a subdirectory named lib.
• Create an empty file named lib\__init__.py.

• In lib\BoxTime.py, write a function foo() like this:

  def foo():
      print "foo!"
  

• In your client code in the directory above lib, write:

  from lib import BoxTime
  BoxTime.foo()
  

• Run your client code. You will get:

  foo!
  

---

Much later -- in linux, it would look like this:

% cd ~/tmp
% mkdir lib
% touch lib/__init__.py
% cat > lib/BoxTime.py << EOF
heredoc> def foo():
heredoc>     print "foo!"
heredoc> EOF
% tree lib
lib
├── BoxTime.py
└── __init__.py

0 directories, 2 files
% python 
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from lib import BoxTime
>>> BoxTime.foo()
foo!

Hazelton

Could you provide a link to the Python documentation where this is explained? Thanks! – Pitta

Let's make that link clickable: docs.python.org/3/tutorial/modules.html#packages – Sidnee

Nice walkthrough for implementing a package lib – Frolicsome

please note: subdirs shall not contain dashes or dots, but underscores are valid. for me that seems like the same restrictions as for other symbol names, but i have not yet digged it down to documentation level. – Southeaster

underscores => python3 (too late for editing the comment) – Southeaster

@AlexanderStohr You are talking about rules for importing? Not something specific to an error I made in my code sample, right? – Hazelton

@Hazelton - i was talking about what will work and even more on what might not work with the argument of "from". – Southeaster

what if i want to select what sub folder to load the module from via a cli argument? from <arg> import module doesn't work – Noisome

Leading zeroes are invalid. For eg, filenames like "000abc.py" don't work – Maniacal

Take care to name your file also BoxTime.py - not boxtime.py etc. if you import with "from lib import BoxTime" - even as my classname was BoxTime it failed when not naming the file accordingly. – Returnee

Answer 3

You can try inserting it in sys.path:

sys.path.insert(0, './lib')
import BoxTime

Gosh

This is great if you for some reason can't or won't create the init.py file. – Twotime

It works if you run python from the "project" directory. The "." is interpreted relative to your current working directory, not relative to the directory where the file you're executing lives. Say you cd /data, python ../project/tester.py. Then it won't work. – Pointillism

This worked for me. I prefer this over an init.py file, it makes for cleaner import statements. – Glaciology

This works MUCH better and is the "correct" solution. init.py messes up packages like boto that have their own child folders with modules. – Jocularity

@Twotime You have to create (at least) a empy file named __init__.py to allow python import modules from that folder. I've tried that solution and works perfectly (v2.7.6). – Krona

This solution works when you are not running an app in the parent folder of the modules you are trying to import and when you don't have the __init__.py in the module folders. – Waterrepellent

@minsk You didn't, by any chance, happen to change the path AFTER attempting the import, instead of before? (I did at first, simply from old habit of beginning with imports.) – Wideranging

A hint if you please, when in troubles with relative path as mentioned by @Pointillism run it like: sys.path.append(sys.path[0] + '/lib') – Shirtwaist

I wish I could give 1000 points for this answer. this answer helped to import a big detectron module. in the project If I havent added sys.path.insert(0, './detectron2') I must have added detectron2 in each file's import line in whole project – Twelvetone

Answer 4

I am writing this down because everyone seems to suggest that you have to create a lib directory.

You don't need to name your sub-directory lib. You can name it anything provided you put an __init__.py into it.

You can do that by entering the following command in a linux shell:

$ touch anything/__init__.py 

So now you have this structure:

$ ls anything/
__init__.py
mylib.py

$ ls
main.py

Then you can import mylib into main.py like this:

from anything import mylib 

mylib.myfun()

You can also import functions and classes like this:

from anything.mylib import MyClass
from anything.mylib import myfun

instance = MyClass()
result = myfun()

Any variable function or class you place inside __init__.py can also be accessed:

import anything

print(anything.myvar)

Or like this:

from anything import myvar

print(myvar)

Deferent

My folder structure is utils\__init__.py and utils\myfile.py. (Utils contain both files) This is how I am trying to import from utils.myfile import myMethod. But I get ModuleNotFoundError: No module named 'utils'. What could be wrong? P.S: I am using Django and trying to import in views.py which is at the same level as utils folder – Farandole

It is possible to use absolute paths when importing modules and run your program with PYTHONPATH=. python path/to/program.py – Deferent

clearest answer! thanks – Dobbin

Answer 5

Try import .lib.BoxTime. For more information read about relative import in PEP 328.

Frayda

I don't think I've ever seen that syntax used before. Is there strong reason (not) to use this method? – Leroi

Why wasn't this the answer. Sure, if you want to do the whole packages thing, you should do that. But that's not what the original question was. – Bolte

This gives me: ValueError: Attempted relative import in non-package – Tiga

This only works if the file you're importing from is itself part of a package. If not, you'll receive the error that @Tiga pointed out. – Cowman

(Tried in Python 3) This results in SyntaxError: invalid syntax. PyLance tells me Relative imports cannot be used with "import .a" form; use "from . import a" instead. In this case this would be from .lib import BoxTime – Hogback

Answer 6

Does your lib directory contain a __init__.py file?

Python uses __init__.py to determine if a directory is a module.

Moonshot

Answer 7

Full example included

This basically covers all cases (make sure you have __init__.py in relative/path/to/your/lib/folder):

import sys, os
sys.path.append(os.path.dirname(os.path.realpath(__file__)) + "/relative/path/to/your/lib/folder")
import someFileNameWhichIsInTheFolder
...
somefile.foo()

Example:

You have in your project folder:

/root/myproject/app.py

You have in another project folder:

/root/anotherproject/utils.py
/root/anotherproject/__init__.py

You want to use /root/anotherproject/utils.py and call foo function which is in it.

So you write in app.py:

import sys, os
sys.path.append(os.path.dirname(os.path.realpath(__file__)) + "/../anotherproject")
import utils

utils.foo()

Prunella

if you're using os.path you probably want to use os.path.join((os.path.dirname(os.path.realpath(__file__)),'..','anotherproject') instead of hardcoding the '/' in your path concatenation. – Duplex

Why can't you just do "../anotherproject" without the os.path.dirname()? – Eats

@MosheRabaev - It is good practice to use os.path functions. In case of wrting "../anotherproject" and moving the code to Windows OS, the code will break! os.path utils knows how to return correct path considering the OS the code running on. for more info docs.python.org/2/library/os.path.html – Prunella

@MosheRabaev and if you use ".." without the dirname(realpath(__file__)), then it will compute the path relative to your current working directory when you run the script, not relative to where the script lives. – Listed

Answer 8

Create an empty file __init__.py in subdirectory /lib. And add at the begin of main code

from __future__ import absolute_import 

then

import lib.BoxTime as BT
...
BT.bt_function()

or better

from lib.BoxTime import bt_function
...
bt_function()

Kyser

This is the only right fix that worked for me in Pycharm, thanks – Fawkes

Answer 9

Just an addition to these answers.

If you want to import all files from all subdirectories, you can add this to the root of your file.

import sys, os
sys.path.extend([f'./{name}' for name in os.listdir(".") if os.path.isdir(name)])

And then you can simply import files from the subdirectories just as if these files are inside the current directory.

Working example

If I have the following directory with subdirectories in my project...

.
├── a.py
├── b.py
├── c.py
├── subdirectory_a
│   ├── d.py
│   └── e.py
├── subdirectory_b
│   └── f.py
├── subdirectory_c
│   └── g.py
└── subdirectory_d
    └── h.py

I can put the following code inside my a.py file

import sys, os
sys.path.extend([f'./{name}' for name in os.listdir(".") if os.path.isdir(name)])

# And then you can import files just as if these files are inside the current directory

import b
import c
import d
import e
import f
import g
import h

In other words, this code will abstract from which directory the file is coming from.

Gemmulation

Hey! I modified your one liner to sys.path.extend([f'{item[0]}' for item in os.walk(".") if os.path.isdir(item[0])]) and now it scrapes recursively from the project's directory. Thank you for your answer! Super useful. – Prevail

@Prevail , Good job! – Gemmulation

Answer 10

For this folder hierarchy diagram example:

/project/tester.py    
/project/lib/BoxTime.py

1- Create a blank py file __init__.py inside lib folder

2- In the caller py file tester.py add theses code lines

import os, sys
sys.path.insert(0,'lib')# insert the folder lib in system path
from BoxTime import Function_name # from the py file import the needed function

Easy explanation can be found in here.

Notice: This is refered to as creating/importing modules in/from different folder.

Personel experience: I tried to create module from jupyter notebook, it did not not work (maybe I done it improperly using .ipynb), I needed to do it manually outside the juypyter notebook, or using other IDE (e.g. pycharm).

Weisbrodt

sys.path.insert was key for me. Thanks! – Lizalizabeth

Answer 11

create_card.py

 init():
   print('Hello world!')

app.py

import create_card

create_card.init()

if you want to import only required functions

from create_card import init

If you have nested directories (Ex: modules/aadhaar/create-card.py)

import modules.aadhaar.create_card as create_card or from modules.aadhaar.create_card import init

Symbol

Answer 12

/project/tester.py

/project/lib/BoxTime.py

create blank file __init__.py down the line till you reach the file

/project/lib/somefolder/BoxTime.py

#lib -- needs has two items one __init__.py and a directory named somefolder #somefolder has two items boxtime.py and __init__.py

Beatrisbeatrisa

Answer 13

try this:

from lib import BoxTime

Newsdealer

without any explanation this isn't very useful. – Bloodfin