What is an alternative to execfile in Python 3?

Question

It seems they canceled in Python 3 all the easy ways to quickly load a script by removing execfile().

Is there an obvious alternative I'm missing?

Paulsen

reload is back, as imp.reload, since 3.2. – Permeability

If you are using Python interactively consider using IPython: %run script_name works with all version of Python. – Reticent

Since 3.4 imp is importlib (which must be imported): importlib.reload(mod_name) imports and executes mod_name. – Wag

what's wrong with runfile("filename.py") ? – Adrianadriana

Thanks @mousomer!! I was precisely looking for the functionality of runfile() since I needed to run a Python script that executes in its own namespace (as opposed to executing on the calling namespace). My application: add the directory of the called script to the system path (sys.path) using the __file__ attribute: if we use execfile() or its equivalent in Python 3 (exec(open('file.py').read())) the included script is run in the calling namespace and thus __file__ resolves to the calling file name. – Jadotville

@TarmoPikaro rolled back your edits, modifying sys.argv is really outside the scope of this function and has some (IMHO) unacceptable down-sides since the caller might be using sys.argv elsewhere. It's possible this could be temporarily overridden (which isn't thread-safe)... but might still be acceptable in some cases. Whatever the case - this wasn't supported by the original execfile, if you want to optionally pass environment, argv, working directory ... etc. This could be a separate answer. – Gladden

Answer 1

According to the documentation, instead of

execfile("./filename") 

Use

exec(open("./filename").read())

See:

• What’s New In Python 3.0
• execfile
• exec

Hedgepeth

Any idea why they would do such a thing? This is so much more verbose than before. Also, it doesn't work for me on Python3.3. I get "No such file or directory" when I exec(open('./some_file').read()). I have tried including the '.py' extension and also excluding the './' as well – Symphonize

Simple and it works as intended for text file. You can even get the output by adding a comma before the last bracket and the name of the output variable. – Glosso

Less trivially, this doesn't provide line numbers when exceptions are raised, as did execfile(). – Lapboard

You'll need to close that file handle too. Another reason to dislike the change from python 2. – Chaulmoogra

How do you reload imported modules within filename ? Without having to exit and re-enter the python environment – Scoff

@Chaulmoogra you don't need to close the file handle in that example, it will be done automatically (at least in regular CPython) – Mcclure

@Mcclure How so? A call to open always needs a follow up call to close docs.python.org/3.6/tutorial/… – Chaulmoogra

@Chaulmoogra in CPython objects are garbage-collected as soon as their reference count goes to 0, only circular references may delay this (stackoverflow.com/questions/9449489/…). In that case that should happen right after read() returns. And file objects are closed on deletion (NB: I realize this link explicitly says "always close files", which is indeed good practice to follow in general) – Mcclure

@Mcclure Interesting, I was under the impression the object wasn't smart enough to close itself. Thanks for opening my eyes to this. I think its still good practice to put a close along side an open call (or better yet use the with statement), you never know how long a variable will linger for, where the code will be used from, nor where the code will end up being pasted. – Chaulmoogra

@Chaulmoogra i always use with because im a forgetful person and 99% of the time forget to close the file, using with eliminates this issue :) – Edwinedwina

how can I pass arguments to this "exec(open("./filename").read())" command ? For example I have a file with data called input_data.csv and want to pass it to script, thanks – Isopod

Some weird edge cases involving executing a file that updates already defined functions and classes can be fixed by including globals() as the second argument for exec – Sextet

with open('./filename') as f: exec(f.read()) deals with opening and closing the file – Naive

@JonathanHartley The question "why do such a thing" remains unanswered. These things do add up. – Chaos

What does the ./ do? In other words, what is the difference (if any) between open("./filename") and open("filename")? – Whisk

@mpb: Nothing: some people don’t understand PATH and think the ./ means “(user-written) program”. – Chihuahua

file isn't reset, so if ./filename depends on using that in its logic, e.g. for activating a virtual environment, this won't work. https://mcmap.net/q/13906/-what-is-an-alternative-to-execfile-in-python-3 is better – Henninger

Answer 2

You are just supposed to read the file and exec the code yourself. 2to3 current replaces

execfile("somefile.py", global_vars, local_vars)

as

with open("somefile.py") as f:
    code = compile(f.read(), "somefile.py", 'exec')
    exec(code, global_vars, local_vars)

(The compile call isn't strictly needed, but it associates the filename with the code object making debugging a little easier.)

See:

• http://docs.python.org/release/2.7.3/library/functions.html#execfile
• http://docs.python.org/release/3.2.3/library/functions.html#compile
• http://docs.python.org/release/3.2.3/library/functions.html#exec

Churchlike

This works for me. However, I noticed that you've written the local and global arguments in the wrong order. It's actually: exec(object[, globals[, locals]]). Of course if you had the arguments flipped in the original, then 2to3 will produce exactly what you said. :) – Jaynajayne

Was pleased to discover that, if you can omit global_vars and local_vars, the python3 replacement here works under python2 as well. Even though exec is a statement in python2, exec(code) works because the parens just get ignored. – Latarsha

+1 for using compile. My "somefile.py" contained inspect.getsourcefile(lambda _: None) which was failing without the compile, because the inspect module couldn't determine where the code was coming from. – Moffit

That's... really ugly. Any idea why they got rid of execfile() in 3.x? execfile also made it easy to pass commandline args. – Impute

open("somefile.py") may be incorrect if somefile.py uses a character encoding different from locale.getpreferredencoding(). tokenize.open() could be used instead. – Loughlin

Another caveat: in python 2, compile() will fail if the source code has trailing whitespace or uses line endings other than '\n'. – Variation

Answer 3

While exec(open("filename").read()) is often given as an alternative to execfile("filename"), it misses important details that execfile supported.

The following function for Python3.x is as close as I could get to having the same behavior as executing a file directly. That matches running python /path/to/somefile.py.

def execfile(filepath, globals=None, locals=None):
    if globals is None:
        globals = {}
    globals.update({
        "__file__": filepath,
        "__name__": "__main__",
    })
    with open(filepath, 'rb') as file:
        exec(compile(file.read(), filepath, 'exec'), globals, locals)

# Execute the file.
execfile("/path/to/somefile.py")

Notes:

• Uses binary file reading to avoid encoding issues.

• Guaranteed to close the file (Python3.x warns about this).

• Defines __main__, some scripts depend on this to check if they are loading as a module or not for eg. if __name__ == "__main__".

• Setting __file__ is nicer for exception messages and some scripts use __file__ to get the paths of other files relative to them.

• Takes optional globals & locals arguments, modifying them in-place as execfile does - so you can access any variables defined by reading back the variables after running.

• Unlike Python2's execfile this does not modify the current namespace by default. For that you have to explicitly pass in globals() & locals().

Gladden

Answer 4

As suggested on the python-dev mailinglist recently, the runpy module might be a viable alternative. Quoting from that message:

https://docs.python.org/3/library/runpy.html#runpy.run_path

import runpy
file_globals = runpy.run_path("file.py")

There are subtle differences to execfile:

• run_path always creates a new namespace. It executes the code as a module, so there is no difference between globals and locals (which is why there is only a init_globals argument). The globals are returned.

  execfile executed in the current namespace or the given namespace. The semantics of locals and globals, if given, were similar to locals and globals inside a class definition.

• run_path can not only execute files, but also eggs and directories (refer to its documentation for details).

Ryun

For some reason, it outputs to the screen a lot of information it was not asked to print ('builtins' etc in Anaconda Python 3). Is there some way to turn this off so that only the information which I output with print() gets visualized? – Depurate

Is it also possible to get all the variables in the current workspace instead of them all being stored in file_globals? This would save having to type the file_globals['...'] for every variable. – Yvetteyvon

"Furthermore, any functions and classes defined by the executed code are not guaranteed to work correctly after a runpy function has returned." Worth noting, depending on your use-case – Wenda

@Yvetteyvon Execute "globals().update(file_globals)". Personally I like this solution best because I can possibly catch errors before deciding to update the current workspace. – Disobedience

@Wenda Thanks for the info, I missed that. Never had any problem like that yet, I wonder what is likely to set that off. – Disobedience

Answer 5

This one is better, since it takes the globals and locals from the caller:

import sys
def execfile(filename, globals=None, locals=None):
    if globals is None:
        globals = sys._getframe(1).f_globals
    if locals is None:
        locals = sys._getframe(1).f_locals
    with open(filename, "r") as fh:
        exec(fh.read()+"\n", globals, locals)

Schizomycete

Actually, this one is the closer to py2 execfile. It even worked for my when using pytests where other solutions posted above failed. Thx! :) – Waggish

Answer 6

You could write your own function:

def xfile(afile, globalz=None, localz=None):
    with open(afile, "r") as fh:
        exec(fh.read(), globalz, localz)

If you really needed to...

Besetting

-1: the exec statment doesn't work this way. Code doesn't run in any version of python. – Vocalize

-1: The default parameter values are evaluated at function definition time, making both globals and locals point to the global namespace fo the module containing the definition of execfile() rather than to the global and local namespace of the caller. The correct approach is to use None as default value and determine the caller's globals and locals via the introspection capabilities of the inspect module. – Tadzhik

Answer 7

If the script you want to load is in the same directory than the one you run, maybe "import" will do the job ?

If you need to dynamically import code the built-in function __ import__ and the module imp are worth looking at.

>>> import sys
>>> sys.path = ['/path/to/script'] + sys.path
>>> __import__('test')
<module 'test' from '/path/to/script/test.pyc'>
>>> __import__('test').run()
'Hello world!'

test.py:

def run():
        return "Hello world!"

If you're using Python 3.1 or later, you should also take a look at importlib.

Bigmouth

This was the correct answer for me. This blog does a good job explaining importlib dev.to/0xcrypto/dynamic-importing-stuff-in-python--1805 – Rolph

This link is dead as I deleted my dev.to account. Reposted at hackberry.xyz/dynamic-importing-stuff-in-python – Tallis

Answer 8

Here's what I had (file is already assigned to the path to the file with the source code in both examples):

execfile(file)

Here's what I replaced it with:

exec(compile(open(file).read(), file, 'exec'))

My favorite part: the second version works just fine in both Python 2 and 3, meaning it's not necessary to add in version dependent logic.

Moffit

Answer 9

Avoid exec() if you can. For most applications, it's cleaner to make use of Python's import system.

This function uses built-in importlib to execute a file as an actual module:

from importlib import util

def load_file_as_module(name, location):
    spec = util.spec_from_file_location(name, location)
    module = util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module

Usage example

Let's have a file foo.py:

def hello():
    return 'hi from module!'
print('imported from', __file__, 'as', __name__)

And import it as a regular module:

>>> mod = load_file_as_module('mymodule', './foo.py')
imported from /tmp/foo.py as mymodule
>>> mod.hello()
hi from module!
>>> type(mod)
<class 'module'>

Advantages

This approach doesn't pollute namespaces or messes with your $PATH whereas exec() runs code directly in the context of the current function, potentially causing name collisions. Also, module attributes like __file__ and __name__ will be set correctly, and code locations are preserved. So, if you've attached a debugger or if the module raises an exception, you will get usable tracebacks.

Note that one minor difference from static imports is that the module gets imported (executed) every time you run load_file_as_module(), and not just once as with the import keyword.

Genip

Great answer! You could extend it by illustrating how calling load_file twice for the same file will or will not reload the file (I wouldn't know without trying). – Mo

@Mo Thanks! I added a note. (It's (re-)executed on every call as the name of loader.exec_module() somewhat suggests.) – Genip

Answer 10

Note that the above pattern will fail if you're using PEP-263 encoding declarations that aren't ascii or utf-8. You need to find the encoding of the data, and encode it correctly before handing it to exec().

class python3Execfile(object):
    def _get_file_encoding(self, filename):
        with open(filename, 'rb') as fp:
            try:
                return tokenize.detect_encoding(fp.readline)[0]
            except SyntaxError:
                return "utf-8"

    def my_execfile(filename):
        globals['__file__'] = filename
        with open(filename, 'r', encoding=self._get_file_encoding(filename)) as fp:
            contents = fp.read()
        if not contents.endswith("\n"):
            # http://bugs.python.org/issue10204
            contents += "\n"
        exec(contents, globals, globals)

Cyanamide

What is "the above pattern"? Please use links when referring to other posts on StackOverflow. Relative positioning terms like "the above" don't work, as there are 3 different ways of sorting answers (by votes, by date, or by activity) and the most common one (by votes) is volatile. Over time your post and posts around yours will end up with different scores, meaning they'll be rearranged and such comparisons will be less useful. – Moffit

Very good point. And given that I wrote this answer almost six months ago, I assume by "above pattern" I meant https://mcmap.net/q/13906/-what-is-an-alternative-to-execfile-in-python-3 (which unfortunately you have to click on to resolve), or better still Noam's answer: – Cyanamide

Generally when I want to refer to other answers to the same question from my answer, I type "Noam's Answer" (for example) and link the text to the answer I'm referring to, just for in case the answer becomes disassociated from the user in the future, IE, because the user changes their account name or the post becomes a communal wiki because too many edits have been made on it. – Moffit

How do you get the URL to a specific "answer" with in a post, excluding the poster's name of the answer? – Swithbart

View the source and get the ID. For example, your question would be stackoverflow.com/questions/436198/… . I'm all for a better method, but don't see anything when I hover near the comment – Cyanamide

Related: Read a unicode file in python which declares its encoding in the same way as python source – Amphibole

@Cyanamide Comment Reply Attempt Not related to answer; just trying out commenting trick mentioned by Eric. Also In Google Chrome right-click-Inspect Element helps find the comment ID. – Swithbart

@Eric: to get a link to an answer: click share link under it. To get a link to a comment, right-click on the timestamp and click "copy link address". – Loughlin

@J.F.Sebastian Noted. – Cyanamide

Answer 11

Also, while not a pure Python solution, if you're using IPython (as you probably should anyway), you can do:

%run /path/to/filename.py

Which is equally easy.

Paulsen

Answer 12

I'm just a newbie here so maybe it's pure luck if I found this :

After trying to run a script from the interpreter prompt >>> with the command

    execfile('filename.py')

for which I got a "NameError: name 'execfile' is not defined" I tried a very basic

    import filename

it worked well :-)

I hope this can be helpful and thank you all for the great hints, examples and all those masterly commented pieces of code that are a great inspiration for newcomers !

I use Ubuntu 16.014 LTS x64. Python 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609] on linux

Heliotropism