python noobie scoping question

Question

I wrote this code:

x = 0
def counter():
 x = 1
 def temp(self):
  print x
  x += 1
 return temp

Trying to test if python is lexical or dynamic scope. My thinking was that

y = counter()
y()

Should either print 0 or 1, and that would tell me how python is scoped. However, calling y throws an exception saying that x is undefined. There seems to be something fundamentally flawed in my understanding of how Python works.

Can someone explain how this works? Yes, I am aware this can easily be done using objects. I'm trying to explore the idea of giving functions state without the use of objects. I wrote the code this way because the above translated to a lexically scoped language like Scheme would definitely have worked.

Viva

possible duplicate of Read/Write Python Closures – Heatstroke

Answer 1

From the docs:

A special quirk of Python is that – if no global statement is in effect – assignments to names always go into the innermost scope. Assignments do not copy data — they just bind names to objects.

So, when Python parses

 def temp(self):
  print x
  x += 1

it sees the assignment x += 1 and therefore decides that x must be in the innermost scope. When later you call temp(...) via y() -- (where, by the way, either the self should be omitted from the definition of temp or else y() should supply one argument) -- Python encounters the print x statement and finds that x has not been defined in the local (innermost) scope. Hence the error,

UnboundLocalError: local variable 'x' referenced before assignment

If you declare

 def temp(self):
     global x

then Python will look for x in the global scope (where x=0). In Python2 there is no way to tell Python to look for x in the extended scope (where x=1). But in Python3 that can be achieved with the declaration

 def temp(self):
     nonlocal x

Stromberg

Answer 2

Python has two scopes (which is really three) plus special rules for nested local scopes. The two scopes are global, for the module-level names, and local, for anything in a function. Anything you assign to in a function is automatically local, unless you declare it otherwise with the global statement in that function. If you use a name that is not assigned to anywhere in the function, it's not a local name; Python will (lexically) search nesting functions to see if they have that name as a local name. If there are no nesting functions or the name isn't local in any of them, the name is assumed to be global.

(The global namespace is special as well, because it's actually both the module global namespace and the builtin namespace, which is tucked away in the builtins or __builtins__ module.)

In your case, you have three x variables: one in the module (global) scope, one in the counter function and one in the temp function -- because += is also an assignment statement. Because the mere fact of assigning to the name makes it local to the function, your += statement will try to use a local variable that hasn't been assigned to yet, and that will raise an UnboundLocalError.

If you intended for all three of these x references to refer to the global variable, you'd need to do global x in both the counter and the temp function. In Python 3.x (but not 2.x), there is a nonlocal declaration quite like global that you can use to make temp assign to the variable in counter, but leave the global x alone.

Innards

Answer 3

Closures in Python aren't writeable, so you can't write your code that way. If you only read from the variable inside the functions, you should be good. In Python 3 you could use the nonlocal keyword to get the behavior you want.

If you are using Python 2.5 or later you could also use the yield keyword to write the above code as a generator.

Patriliny

Answer 4

The Python documentation answers the question in detail: http://docs.python.org/reference/executionmodel.html#naming-and-binding

In summary, Python has static scoping rules. If function f defines or deletes a variable name, then the variable name refers to a variable in the closure of function f. If function f only uses a variable name (no definition or deletion), then the name refers to whatever the name means in f's parent scope. Keep going up to parent scopes until a definition is found or you reach global scope. For example:

def f1(x):
  def g(y):
    z.foo = y # assigns global z
  g(1)

def f2(x):
  def g(y):
    x.foo = y # assigns f2's variable, because f2 defines x 
  g(1)

def f3(x):
  def g(y):
    x = C()
    x.foo = y # assigns g's variable, because g defines x
  g(1)

The global keyword and (in Python 3) the nonlocal keywords override the default scoping rules.

While variable names are resolved statically, variable values are resolved dynamically. The value of the variable comes from the most recent definition or deletion of that variable at the time the variable is accessed. Values are looked up in the function closure where the variable resides.

Forelady

Whatever the Python folks may call it, a variable that you forget to initialize in a Python function has dynamic scope. This is one of the more glaring warts of Python (not necessarily that it's dynamic scope, but that it's sometimes dynamic scope). – Assyriology

@ChrisPacejo: In what sense is that "dynamic scope" (as opposed to lexical scope)? – Disbelieve

Answer 5

Thanks for all your replies. Just in case anyone cares, I've sort of come up with a workaround for this problem. I did it by creating a "scoper" function to establish a counter variable.

>>> def gc():
...  def scoper():
...   scoper.s = 0
...   def rtn():
...    scoper.s += 1
...    return scoper.s
...   return rtn
...  return scoper()

The above allowed me to do this, which mimics proper closures:

>>> a = gc()
>>> a()
1
>>> a()
2
>>> a()
3
>>> b = gc()
>>> b()
1
>>> a()
4
>>> b()
2

Viva