Calling locals() in a function not intuitive?
Asked Answered
K

4

7

This may be elementary, but may help me understand namespaces. A good explanation might step through what happens when the function definition is executed, and then what happens later when the function object is executed. Recursion may be complicating things.

The results aren't obvious to me; I would have expected:

locals_1 would contain var; locals_2 would contain var and locals_1; and locals_3 would contain var, locals_1, and locals_2

# A function calls locals() several times, and returns them ...
def func():
  var = 'var!'
  locals_1 = locals()
  locals_2 = locals()
  locals_3 = locals()
  return locals_1, locals_2, locals_3

# func is called ...
locals_1, locals_2, locals_3 = func()

# display results ...
print 'locals_1:', locals_1
print 'locals_2:', locals_2
print 'locals_3:', locals_3

Here are the results:

locals_1: {'var': 'var!', 'locals_1': {...}, 'locals_2': {...}}
locals_2: {'var': 'var!', 'locals_1': {...}, 'locals_2': {...}}
locals_3: {'var': 'var!', 'locals_1': {...}, 'locals_2': {...}}

The pattern seems to be, with (n) calls to locals, all of
the returned locals-dicts are identical, and they all include the first (n-1) locals-dicts.

Can someone explain this?

More specifically:

Why does locals_1 include itself?

Why does locals_1 include locals_2? Is locals_1 assigned when func is created, or executed?

And why is locals_3 not included anywhere?

Does "{...}" indicate an 'endless recursion'? Sort of like those photos of mirrors facing each other?

Keldah answered 28/4, 2014 at 15:28 Comment(0)
K
0

My original question settles down to, 'just what is locals()?' Here's my current (speculative) understanding, written in Pythonese:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

What is the nature of Python's locals() ?

Every local namespace has it's own namespace-table, which can be viewed in total using the locals built-in function. (A namepace-table, in effect, is like a dict that holds "identifier": object entries; for each item, the key is the name (in string-form) assigned (or 'bound') to the object.)

When called in a non-global level, locals returns the interpreter's sole representation of the current local namespace-table: a 'dynamic', always-up-to-date, specialized, dict-like object.

It's not a simple dict, nor is it the actual name-table, but it's effectively 'alive', and is instantly updated from the live table anytime it is referenced (when tracing is on, it updates with every statement).
On exiting the scope, this object vanishes, and is created anew for the current scope whenever locals is next called.

(When called in a global (modular) level, locals instead returns globals(), Python's global-namespace representation, which may have a different nature).

So, L = locals() binds the name L to this 'stand-in' for the local namespace-table; subsequently, anytime L is referenced, this object is refreshed and returned.
And, any additional names bound (in the same scope) to locals() will be aliases for this same object.

Note that L, being assigned to locals(), necessarily becomes an 'infinite recursion' object (displayed as the dict {...}), which may or may not be important to you. You can make a simple dict copy of L at any time, however.
Some attributes of locals(), such as keys, also return simple objects.

To capture a 'pristine' snapshot of locals() within a function, use a technique that doesn't make any local assignments; for example, pass the copy as argument to a function, that pickles it out to a file.

There are particulars about L, and how it behaves; it includes free variables from the blocks of functions, but not classes, and, the docs warn against trying to alter L's contents (it may no longer 'mirror' the name-table).
It perhaps should only be read (copied, etc.)

(Why locals() is designed to be 'live', rather than a 'snapshot', is another topic).

In summary:

locals() is a unique, specialized object (in dict-form); it's Python's live representation of the current local namespace-table (not a frozen snapshot)

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

A way to get the results I had expected, then, is to produce copies of locals() (here using dict.copy), at each step:

# A function copies locals() several times, and returns each result ...
def func():
    var = 'var!'
    locals_1 = locals().copy()
    locals_2 = locals().copy()
    locals_3 = locals().copy()
    return locals_1, locals_2, locals_3

func is called, and the returns are displayed:

locals_1: {'var': 'var!'}
locals_2: {'var': 'var!', 'locals_1': {'var': 'var!'}}
locals_3: {'var': 'var!', 'locals_1': {'var': 'var!'}, 'locals_2':{'var':'var!','locals_1': {'var': 'var!'}}}

The returns are simple dict objects, that capture the growing stages of the local namespace.
This is what I intended.

Other possible ways to copy locals() (here "L") are dict(L), copy.copy(L) and copy.deepcopy(L).

Keldah answered 5/5, 2014 at 17:31 Comment(0)
T
5

Let's run this code:

def func():
  var = 'var!'
  locals_1 = locals()
  print(id(locals_1), id(locals()), locals())
  locals_2 = locals()
  print(id(locals_2), id(locals()), locals())
  locals_3 = locals()
  print(id(locals_3), id(locals()), locals())
  return locals_1, locals_2, locals_3


func()

This would be in the output:

44860744 44860744 {'locals_1': {...}, 'var': 'var!'}
44860744 44860744 {'locals_2': {...}, 'locals_1': {...}, 'var': 'var!'}
44860744 44860744 {'locals_2': {...}, 'locals_3': {...}, 'locals_1': {...}, 'var': 'var!'}

locals() here grows as expected, however you are assigning the reference to locals(), not the value of locals() to each variable.

After each assignment locals() gets changed, but reference does not, so each variable is pointing to the same object. In my output all object ids are equal, this is the proof.

Longer explanation

These variables has the same link (reference) to that object. Basically, all variables in Python are references (similar concept to pointers).

        locals_1            locals_2                 locals_3
            \                    |                      /
             \                   |                     /
              V                  V                    V
            ---------------------------------------------
            |            single locals() object         |
            ---------------------------------------------

They have absolutely no idea what value does locals() have, they only know where to get it when it's needed (when the variable is used somewhere). Changes on locals() don't affect those variables.

In the end of you function you're returning three variables, and this is what happening when you're printing them:

print(locals_N) -> 1. Get object referenced in locals_N
                   2. Return the value of that object

See? So, this is why they have exactly the same value, the one locals() has at the moment of print.

If you change locals() (somehow) again and then run print statements, what would be printed 3 times? Yep, the NEW value of locals().

Tiptoe answered 28/4, 2014 at 15:32 Comment(7)
I'll have to study this, great, but I can still ask: why does locals_1 contain itself - I might expect it wouldn't?Keldah
@tfj, as @frostnational says, locals_1 is not actually a separate dictionary from the one returned by locals(). Rather, it is a reference to the same dictionary. Changes to locals() will be reflected in locals_1. So when local_1 is added to the scope, locals_1 gains an entry that refers to itself.Stingy
@bgschiller, is "reference" the same thing as "identifier" (commonly, "name")? But why does locals() capture itself on the first try? By the time it's created, it's too late (I would have thought).Keldah
@Keldah locals_1 is a reference to locals() object. Then, locals() contains reference to itself and this is indeed an endless recursion.Tiptoe
@Keldah "reference" is a similar idea to "pointer". You can think of locals() and locals_1 as 'pointing' to the very same object in memory. So, when the value of locals() changes, locals_1 gets the update automatically. There's no time-travel happening -- everything happens in the order you expect. It's just that by the time you go to print the value of locals_1, it looks different.Stingy
I think I see: by the time the func returns, all three 'variables'Keldah
... (continuing) have been updated, in fact they all name the same object ("identifier" is an official Python term).Keldah
H
4

I like your question, really good one.

Yes, locals() is sort of magic, but with your approach, you will get it soon or later and will love it.

Key concepts

Dictionary is assigned by reference and not by value

In [1]: a = {"alfa": 1, "beta": 2}

In [2]: b = a

In [3]: b
Out[3]: {'alfa': 1, 'beta': 2}

In [4]: b["gama"] = 3

In [5]: b
Out[5]: {'alfa': 1, 'beta': 2, 'gama': 3}

In [6]: a
Out[6]: {'alfa': 1, 'beta': 2, 'gama': 3}

As you see, a get changed indirectly at the moment b got modified, because both a and b are pointing to the same data structure in memory.

locals() is returning dictionary with all local variables

Edit: clarified when is this dict updated

So all local variables existing at the moment of locals() call are living here. If you make subsequent calls to locals(), this dictionary is updated at the moment of the call.

Answering your questions

Why does locals_1 include itself?

Because locals_1 is reference to dictionary of all locally defined variables. As soon as locals_1 becames part of local namespace, it gets part of the dictionary locals() returned.

Why does locals_1 include locals_2? Is locals_1 assigned when func is created, or executed?

The same answer as previous one applies.

And why is locals_3 not included anywhere?

This is the most difficult part of your question. After some research I found excellent article about this topic: http://nedbatchelder.com/blog/201211/tricky_locals.html

The fact is, that locals() returns a dictionary, which contains references to all local variables. But the tricky part is, it is not directly that structure, it is a dictionary, which is only updated at the moment, locals() is called.

This explains missing locals_3 in your result. All results are pointing to the same dictionary, but it does not get updated after you introduce locals_3 variable.

When I added another print locals() before return, I found it there, without it not.

Uff.

Does "{...}" indicate an 'endless recursion'? Sort of like those photos of mirrors facing each other?

I would read it as "there is something more". But I think, you are right, that this is a solution for printing recursive data structure. Without such a solution the dictionary could not be really printed in finite amount of time.

Bonus - use of **locals() in string.format()

There is one idiom, where locals() are shortening your code a lot, in string.format()

name = "frost"
surname = "national"
print "The guy named {name} {surname} got great question.".format(**locals())
Hereafter answered 28/4, 2014 at 17:49 Comment(1)
Just make sure you disable warning 0142 in Pylint if you use that last one, since it will complain about your use of "black magic"!Agrapha
P
1

frostnational and Jan Vlcinsky have already given good explanations of what happens behind the scenes. Here's a small addition on achieving the behavior that you originally expected. You can use the copy method to create a copy of the locals() dict. That copy will not be updated when locals() is updated, so it contains the "snapshot" that you expected:

In [1]: def func():
   ...:     var = 'var!'
   ...:     locals1 = locals().copy()
   ...:     locals2 = locals().copy()
   ...:     locals3 = locals().copy()
   ...:     return locals1, locals2, locals3
   ...:

In [2]: locals1, locals2, locals3 = func()

In [3]: locals1
Out[3]: {'var': 'var!'}

In [4]: locals2
Out[4]: {'locals1': {'var': 'var!'}, 'var': 'var!'}

In [5]: locals3
Out[5]:
{'locals1': {'var': 'var!'},
 'locals2': {'locals1': {'var': 'var!'}, 'var': 'var!'},
 'var': 'var!'}

As expected, each copy only contains the variables that have been defined before locals() is called.

Pinprick answered 29/4, 2014 at 9:42 Comment(0)
K
0

My original question settles down to, 'just what is locals()?' Here's my current (speculative) understanding, written in Pythonese:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

What is the nature of Python's locals() ?

Every local namespace has it's own namespace-table, which can be viewed in total using the locals built-in function. (A namepace-table, in effect, is like a dict that holds "identifier": object entries; for each item, the key is the name (in string-form) assigned (or 'bound') to the object.)

When called in a non-global level, locals returns the interpreter's sole representation of the current local namespace-table: a 'dynamic', always-up-to-date, specialized, dict-like object.

It's not a simple dict, nor is it the actual name-table, but it's effectively 'alive', and is instantly updated from the live table anytime it is referenced (when tracing is on, it updates with every statement).
On exiting the scope, this object vanishes, and is created anew for the current scope whenever locals is next called.

(When called in a global (modular) level, locals instead returns globals(), Python's global-namespace representation, which may have a different nature).

So, L = locals() binds the name L to this 'stand-in' for the local namespace-table; subsequently, anytime L is referenced, this object is refreshed and returned.
And, any additional names bound (in the same scope) to locals() will be aliases for this same object.

Note that L, being assigned to locals(), necessarily becomes an 'infinite recursion' object (displayed as the dict {...}), which may or may not be important to you. You can make a simple dict copy of L at any time, however.
Some attributes of locals(), such as keys, also return simple objects.

To capture a 'pristine' snapshot of locals() within a function, use a technique that doesn't make any local assignments; for example, pass the copy as argument to a function, that pickles it out to a file.

There are particulars about L, and how it behaves; it includes free variables from the blocks of functions, but not classes, and, the docs warn against trying to alter L's contents (it may no longer 'mirror' the name-table).
It perhaps should only be read (copied, etc.)

(Why locals() is designed to be 'live', rather than a 'snapshot', is another topic).

In summary:

locals() is a unique, specialized object (in dict-form); it's Python's live representation of the current local namespace-table (not a frozen snapshot)

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

A way to get the results I had expected, then, is to produce copies of locals() (here using dict.copy), at each step:

# A function copies locals() several times, and returns each result ...
def func():
    var = 'var!'
    locals_1 = locals().copy()
    locals_2 = locals().copy()
    locals_3 = locals().copy()
    return locals_1, locals_2, locals_3

func is called, and the returns are displayed:

locals_1: {'var': 'var!'}
locals_2: {'var': 'var!', 'locals_1': {'var': 'var!'}}
locals_3: {'var': 'var!', 'locals_1': {'var': 'var!'}, 'locals_2':{'var':'var!','locals_1': {'var': 'var!'}}}

The returns are simple dict objects, that capture the growing stages of the local namespace.
This is what I intended.

Other possible ways to copy locals() (here "L") are dict(L), copy.copy(L) and copy.deepcopy(L).

Keldah answered 5/5, 2014 at 17:31 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.