How to get local variables updated, when using the `exec` call?

Question

I thought this would print 3, but it prints 1:

def f():
    a = 1
    exec("a = 3")
    print(a)

f()

I'm using Python 3. In Python 2, it works as expected (prints 3).

Duwalt

Because a is already defined in this case we can use python - Modify existing variable in locals() or frame.f_locals - Stack Overflow – Cachexia

Answer 1

This issue is somewhat discussed in the Python3 bug list. Ultimately, to get this behavior, you need to do:

def foo():
    ldict = {}
    exec("a = 3", globals(), ldict)
    a = ldict['a']
    print(a)

And if you check the Python3 documentation on exec, you'll see the following note:

The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns.

That means that one-argument exec can't safely perform any operations that would bind local variables, including variable assignment, imports, function definitions, class definitions, etc. It can assign to globals if it uses a global declaration, but not locals.

Referring back to a specific message on the bug report, Georg Brandl says:

To modify the locals of a function on the fly is not possible without several consequences: normally, function locals are not stored in a dictionary, but an array, whose indices are determined at compile time from the known locales [sic]. This collides at least with new locals added by exec.

Then, regarding Python 2:

The old exec statement circumvented this, because the compiler knew that if an exec without globals/locals args occurred in a function, that namespace would be "unoptimized", i.e. not using the locals array. Since exec() is now a normal function, the compiler does not know what "exec" may be bound to, and therefore can not treat is specially.

Emphasis is mine.

So the gist of it is that Python3 can better optimize the use of local variables by not allowing this behavior by default.

And for the sake of completeness, this does work as expected in Python 2.X:

Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) 
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def f():
...     a = 1
...     exec "a=3"
...     print a
... 
>>> f()
3

Weisberg

I see, it's an issue with locals() that was hacked out of exec in python 2.X. This issue is not as clearly documented as I would have liked. Exec/locals changing from 2.X to 3.X should be pointed out somewhere docs.python.org/3.1/library/functions.html#exec and I think exec should have a a convenience parameter that circumvents this optimization... – Duwalt

@AnatolyAlekseev: It's documented (in the sense of "Modifications to default locals are unsupported"), and there is no good fix that doesn't involve either restoring exec to the status of a keyword, performance regressions in code that doesn't need this feature, or performing really kludgy stuff to make writes to locals pass through to the "real" locals (which might not be practical in non-CPython interpreters). Point is, exec is, and always has been, a bad idea, and in the rare cases you need to achieve the functionality described, there are workarounds (as described in this answer). – Cherimoya

@AnatolyAlekseev In addition to what's been said already, another problem is, the damage has already been done. Hypothetically if Python went back to the old exec behavior in e.g,. Python 3.8 or whatever, then scripts that were written between Python 2.7 and Python 3.8 may have assumed that exec was provided its own scope, and they did calculations assuming that writing to some variable i in exec would not affect the global scope. It probably shouldn't have been changed but changing it again now would just make more problems. – Hyper

@Hyper IMO it's great this was done, this should have never been allowed in the first place. The more barriers to people who want to do these sorts of things the better. If in the highly specialized cases where this might actually be needed or be reasonable, then there are workarounds. – Tucker

Answer 2

If you are inside a method, you can do so:

# python 2 or 3
class Thing():
    def __init__(self):
        exec('self.foo = 2')
    
x = Thing()
print(x.foo)

You can read more about it here

Guinevere

ref bugs.python.org/issue4831 – Necrophilism

But self.foo isn't a local, it's an attribute. Is this meant to be a workaround? – Tack

Answer 3

The reason that you can't change local variables within a function using exec in that way, and why exec acts the way it does, can be summarized as following:

1. exec is a function that shares its local scope with the scope of the most inner scope in which it's called.
2. Whenever you define a new object within a function's scope it'll be accessible in its local namespace, i.e. it will modify the local() dictionary. When you define a new object in exec what it does is roughly equivalent to following:

---

from copy import copy
class exec_type:
    def __init__(self, *args, **kwargs):
        # default initializations
        # ...
        self.temp = copy(locals())

    def __setitem__(self, key, value):
        if var not in locals():
            set_local(key, value)
        self.temp[key] = value

temp is a temporary namespace that resets after each instantiation (each time you call the exec).

---

3. Python starts looking up for the names from local namespace. It's known as LEGB manner. Python starts from Local namespce then looks into the Enclosing scopes, then Global and at the end it looks up the names within Buit-in namespace.

A more comprehensive example would be something like following:

g_var = 5

def test():
    l_var = 10
    print(locals())
    exec("print(locals())")
    exec("g_var = 222")
    exec("l_var = 111")
    exec("print(locals())")

    exec("l_var = 111; print(locals())")

    exec("print(locals())")
    print(locals())
    def inner():
        exec("print(locals())")
        exec("inner_var = 100")
        exec("print(locals())")
        exec("print([i for i in globals() if '__' not in i])")

    print("Inner function: ")
    inner()
    print("-------" * 3)
    return (g_var, l_var)

print(test())
exec("print(g_var)")

Output:

{'l_var': 10}
{'l_var': 10}

locals are the same.

{'l_var': 10, 'g_var': 222}

after adding g_var and changing the l_var it only adds g_var and left the l_var unchanged.

{'l_var': 111, 'g_var': 222}

l_var is changed because we are changing and printing the locals in one instantiation ( one call to exec).

{'l_var': 10, 'g_var': 222}
{'l_var': 10, 'g_var': 222}

In both function's locals and exec's local l_var is unchanged and g_var is added.

Inner function: 
{}
{'inner_var': 100}
{'inner_var': 100}

inner_function's local is same as exec's local.

['g_var', 'test']

global is only contain g_var and function name (after excluding the special methods).

---------------------

(5, 10)
5

Whisk