Python imports being overridden by the standard library (Python 2.4)

Question

I have a python package that I'm writing and I'm having an issue where the standard library is being imported instead of my files because of name clashes.

For example, a file structure like below:

package/__init__.py

# No data in this file

package/module.py

#!/usr/bin/env python
print 'Loading module.py'
import signal

package/signal.py

#!/usr/bin/env python
print 'Loading signal.py'

I get the following results when I run this:

$ ./module.py
Loading module.py

I would like to get:

$ ./module.py
Loading module.py
Loading signal.py

The actual question:

So, when I run module.py, it's import signal goes to the stdlib version. How am I able to force module.py to import signal.py instead?

As noted in the tags, this needs to be able to run on python-2.4.3. While that is an old version, it's what is included in RHEL 5.

---

Some additional information

Just for a bit more information, I explicitly have the following setup:

[10:30pm][~/test] tree .
.
|-- package
|   |-- __init__.py
|   |-- module.py
|   `-- signal.py
`-- script

[10:30pm][~/test] cat script
#!/usr/bin/env python
from package import signal

[10:30pm][~/test] cat package/__init__.py

[10:30pm][~/test] cat package/module.py 
#!/usr/bin/env python
print "Loading module.py"
import signal

[10:30pm][~/test] cat package/signal.py 
#!/usr/bin/env python
print "Loading signal.py"

[10:30pm][~/test] python ./script
Loading signal.py

[10:32pm][~/test] python ./package/module.py 
Loading module.py

[10:32pm][~/test] python -m package.module
python: module package.module not found

Please note that when I ran ./package/module.py that the print statement in ./package/signal.py was not fired. This implies that the signal that was loaded is the one from the stdlib.

Brasca

What's wrong with renaming your packages to avoid a name clash? – Control

@S.Lott: While in this example it does conflict with something in the standard library, won't this also occur if the user installs an egg that clashes? – Brasca

Don't use RHEL :). ./configure; make; make install. Blau – Hypophosphate

There is nothing wrong with RHEL. Your suggestion should recommend make altinstall, since upgrading python will trash a RHEL 5 system, which requires 2.4 to be /usr/bin/python or yum will stop functioning. A better alternative is to get Python 2.6 from the EPEL. He doesn't need to upgrade python, so why should he change from a stable OS to do so? – Slagle

But I get the following: $ python2 -m package.module => Loading module.py Loading signal.py – Eppes

@FelixYan: Are you running python 2.4? – Brasca

@Felix Yan: -m works properly only on Python2.6+ docs.python.org/whatsnew/… – Pardoner

Answer 1

The problem in this case is that the built-in signal module is imported as part of the interpreter's startup process. So by the time your code in module.py runs, there is already an entry in sys.modules under the name signal, with the built-in module as its value. Since sys.modules is the first place the import statement looks, it's going to return that built-in module and stop there. It doesn't even bother to look for your own custom signal.py.

I can think of three solutions:

1. Rename your module. This is the easiest solution. You don't have to worry about your module names clashing with installed packages (eggs), because normally the import statement will search directories in the order specified by sys.path, and the current working directory when you run the interpreter is first on that list. signal is sort of a special case because it gets imported automatically before your code runs, but there are a limited number of modules for which that is true, and those are the only ones you have to avoid name clashes with.
2. You can del sys.modules['signal'] and replace the entry with your own custom signal module. Don't actually do this unless your module is truly meant to be a drop-in replacement for the built-in signal. If you do, then any code that runs import signal from that point on will get your version, not the built-in one, and that could wreak havoc on any internal code that needs the signal module to function.

3. You can use the imp module to get access to the internals of the import function, or at least equivalent code, which will allow you to bypass the check of sys.modules.

   If you really do need to do this, here's a code sample you could use:

   import imp, sys
   
   f, path, desc = imp.find_module('signal', sys.path)
   if f:
       f.close()
       signal = imp.new_module('signal')
       execfile(path, signal.__dict__)
   else:
       raise ImportError('signal.py not found')
   # signal is your module
   

   This code snippet is roughly equivalent to import signal except that it does not insert the module found into sys.modules, and it does not look for builtin modules before searching the path.

Housing

That's really quite lame. Thank you for the explanation. – Brasca

Answer 2

From the project directory (a parent of package directory):

python -mpackage.state # Python 2.6+
python2.4 -c'from package.state import test; test()'

• always use absolute imports in your modules i.e., import package.signal instead of import signal unless you know what you're doing
• make sure that package's parent directory is in sys.path. unittest2, nose, pytest test runners might do it for you
• avoid module names that conflict with stdlib modules. Many test runners, code analyzers are broken and add directories with a module being processed to sys.path
• python package/stat.py adds package to sys.path (see http://bugs.python.org/issue13475). You don't want that in this case

Pardoner

Answer 3

File Hierarchy

.
|-- package
|   |-- __init__.py
|   |-- module.py
|   `-- signal.py
`-- script
`-- __init__.py

script.py

#!/usr/bin/env python
from package import module

package/module.py

#!/usr/bin/env python
print 'Loading module.py'
from package import signal

package/signal.py

#!/usr/bin/env python
print 'Loading signal.py'

Output:

$ ./script 
Loading module.py
Loading signal.py

Edit

Just tested this with a modification to the module.py, on python 2.4.3

Dyal

@sharth - Sorry. Just made a change and tested it on python 2.4.3 – Dyal

I've updated the gist with new output. I agree that ./script will run. That has always been the case. I'd also like to be able to execute ./package/signal.py. – Brasca