Python package structure

Question

I have a Python package with several subpackages.

myproject/
  __init__.py
  models/
    __init__.py
    ...
  controllers/
    __init__.py
    ..
  scripts/
    __init__.py
    myscript.py

Within myproject.scripts.myscript, how can I access myproject.models? I've tried

from myproject import models # No module named myproject
import models # No module named models
from .. import models # Attempted relative import in non-package

I've had to solve this before, but I can never remember how it's supposed to be done. It's just not intuitive to me.

Auxochrome

How are you running myscript.py? Could you show us your command line? That will make a difference to the answer. – Alejandraalejandrina

Try import myproject.models – Supersensitive

Okay, I found a solution: sys.path.append(os.path.dirname(os.path.dirname(__file__))) before import. I hope there is a better solution though. – Auxochrome

It's probably got to do with your pythonpath. Where are you trying to import this package from? If it's not in the same location as the package you have to add it to the pythonpath. – Batey

Brandon Craig Rhodes: I am in the myproject/ directory, and I am running python scripts/myscript.py. – Auxochrome

@adeel: did you try import myproject.models? – Supersensitive

John: Yes, that works only if I set PYTHONPATH as Jean-Paul Calderone mentioned in his answer. – Auxochrome

Ok then. I guess Jean-Paul gets an upvote from me. Glad you got it working. – Supersensitive

Answer 1

This is the correct version:

from myproject import models

If it fails with ImportError: No module named foo it is because you haven't set PYTHONPATH to include the directory which contains myproject/.

I'm afraid other people will suggest tricks to let you avoid setting PYTHONPATH. I urge you to disregard them. This is why PYTHONPATH exists: to tell Python where to look for code to load. It is robust, reasonably well documented, and portable to many environments. Tricks people play to avoid having to set it are none of these things.

The explicit relative import will work even without PYTHONPATH being set, since it can just walk up the directory hierarchy until it finds the right place, it doesn't need to find the top and then walk down. However, it doesn't work in a script you pass as a command line argument to python (or equivalently, invoke directly with a #!/usr/bin/python line). This is because in both these cases, it becomes the __main__ module of the process. There's nowhere to walk up to from __main__ - it's already at the top! If you invoke the code in your script by importing that module, then it will be fine. That is, compare:

python myproject/scripts/myscript.py

to

python -c 'import myproject.scripts.myscript'

You can take advantage of this by not executing your script module directly, but creating a bin/myscript that does the import and perhaps calls a main function:

import myprojects.scripts.myscript
myprojects.scripts.myscript.main()

Compare to how Twisted's command line scripts are defined: http://twistedmatrix.com/trac/browser/trunk/bin/twistd

Gorky

Thanks. Can you explain why from .. import models doesn't work? I thought relative imports would allow you to do this without setting PYTHONPATH. – Auxochrome

Thanks, Jean-Paul! I like that approach. – Auxochrome

Answer 2

Your project is not in your path.

Option A

• Install your package so that python can find it via its absolute name from anywhere (using from myproject import models )

Option B

• Trickery to add the relative parent to your path
• sys.path.append(os.path.abspath('..'))

The former option is recommended.

Edee