Python nested functions variable scoping [duplicate]

Question

I've read almost all the other questions about the topic, but my code still doesn't work.

I think I'm missing something about python variable scope.

Here is my code:

PRICE_RANGES = {
                64:(25, 0.35),
                32:(13, 0.40),
                16:(7, 0.45),
                8:(4, 0.5)
                }

def get_order_total(quantity):
    global PRICE_RANGES
    _total = 0
    _i = PRICE_RANGES.iterkeys()
    def recurse(_i):
        try:
            key = _i.next()
            if quantity % key != quantity:
                _total += PRICE_RANGES[key][0]
            return recurse(_i) 
        except StopIteration:
            return (key, quantity % key)

    res = recurse(_i)

And I get

"global name '_total' is not defined"

I know the problem is on the _total assignment, but I can't understand why. Shouldn't recurse() have access to the parent function's variables?

Can someone explain to me what I'm missing about python variable scope?

Durrace

This is not an answer to your actual question, but just a note that your whole function can be written as return sum(lower for (key, (lower, upper)) in PRICE_RANGES.iteritems() if quantity % key != quantity) – Rrhoea

Actually, upon closer inspection, you return the (key, quantity % key), so the part I posted is just the way to calculate _total. Your function always seems to return the last key visited - you may think that the order of your dictionary items will be preserved, but they aren't so the final return value of your function is somewhat random. – Rrhoea

Answer 1

When I run your code I get this error:

UnboundLocalError: local variable '_total' referenced before assignment

This problem is caused by this line:

_total += PRICE_RANGES[key][0]

The documentation about Scopes and Namespaces says this:

A special quirk of Python is that – if no global statement is in effect – assignments to names always go into the innermost scope. Assignments do not copy data — they just bind names to objects.

So since the line is effectively saying:

_total = _total + PRICE_RANGES[key][0]

it creates _total in the namespace of recurse(). Since _total is then new and unassigned you can't use it in the addition.

Infancy

Answer 2

In Python 3, you can use the nonlocal statement to access non-local, non-global scopes.

The nonlocal statement causes a variable definition to bind to a previously created variable in the nearest scope. Here are some examples to illustrate:

def sum_list_items(_list):
    total = 0

    def do_the_sum(_list):
        for i in _list:
            total += i

    do_the_sum(_list)

    return total

sum_list_items([1, 2, 3])

The above example will fail with the error: UnboundLocalError: local variable 'total' referenced before assignment

Using nonlocal we can get the code to work:

def sum_list_items(_list):
    total = 0

    def do_the_sum(_list):

        # Define the total variable as non-local, causing it to bind
        # to the nearest non-global variable also called total.
        nonlocal total

        for i in _list:
            total += i

    do_the_sum(_list)

    return total

sum_list_items([1, 2, 3])

But what does "nearest" mean? Here is another example:

def sum_list_items(_list):

    total = 0

    def do_the_sum(_list):

        # The nonlocal total binds to this variable.
        total = 0

        def do_core_computations(_list):

            # Define the total variable as non-local, causing it to bind
            # to the nearest non-global variable also called total.
            nonlocal total

            for i in _list:
                total += i

        do_core_computations(_list)

    do_the_sum(_list)

    return total

sum_list_items([1, 2, 3])

In the above example, total will bind to the variable defined inside the do_the_sum function, and not the outer variable defined in the sum_list_items function, so the code will return 0. Note that it is still possible to do double nesting such as this: if total is declared nonlocal in do_the_sum the above example would work as expected.

def sum_list_items(_list):

    # The nonlocal total binds to this variable.
    total = 0

    def do_the_sum(_list):

        def do_core_computations(_list):

            # Define the total variable as non-local, causing it to bind
            # to the nearest non-global variable also called total.
            nonlocal total

            for i in _list:
                total += i

        do_core_computations(_list)

    do_the_sum(_list)

    return total

sum_list_items([1, 2, 3])

In the above example the nonlocal assignment traverses up two levels before it locates the total variable that is local to sum_list_items.

Tadtada

This is a really important contribution! I didn't know about this statement. Thanks for pointing it out. Maybe you could expand your answer to include an example. – Bobbie

You are welcome to edit it if you would like to. – Tadtada

This is really helpfull. nonlocal worked for me – Materiel

This is really the correct answer now. Python 2 is dead as of the beginning of 2020, so Python3-specific answers are applicable to pretty much everyone now. – Bitterroot

This should be the marked answer – Taxexempt

this is the right answer – Reitman

What about recurssion in such, a scope ? does each level of recurssion increases the need for locality? – Trichromat

UnboundLocalError: local variable 'total'. Here total is 'readable' but can't be modified actually. – Amphitrite

Most importantly you should address why the op's code works if the variable is a list type but does not work if it is a string or int type. – Globigerina

I don't intend to edit an answer for a question that was closed 4 years ago. – Tadtada

but if i need to add the nonlocal keyword, does the LEGB rule even apply anymore? i mean, it should by default look for total in the enclosed scope – Eagleeyed

Answer 3

Here's an illustration that gets to the essence of David's answer.

def outer():
    a = 0
    b = 1

    def inner():
        print a
        print b
        #b = 4

    inner()

outer()

With the statement b = 4 commented out, this code outputs 0 1, just what you'd expect.

But if you uncomment that line, on the line print b, you get the error

UnboundLocalError: local variable 'b' referenced before assignment

It seems mysterious that the presence of b = 4 might somehow make b disappear on the lines that precede it. But the text David quotes explains why: during static analysis, the interpreter determines that b is assigned to in inner, and that it is therefore a local variable of inner. The print line attempts to print the b in that inner scope before it has been assigned.

Agonize

+1 I was confused, but now I see what happens. I'm a c# programmer and every time I begin to like Python something like this comes up and ruins it for me. – Laurettelauri

There's a very good reason why Python works like this. – Comitative

@Comitative can you elaborate on that comment? – Brame

@Brame Oops, I think I commented on the wrong answer by accident. I don't remember where I meant to put the comment. Anyway, I don't know how compelling this is, but the issue is that b would otherwise be changing scope from "global" to "local" within the function. Not only does Python not seem to support this at a low level, but it would be confusing and unnecessary. – Comitative

Answer 4

Rather than declaring a special object or map or array, one can also use a function attribute. This makes the scoping of the variable really clear.

def sumsquares(x,y):
  def addsquare(n):
    sumsquares.total += n*n

  sumsquares.total = 0
  addsquare(x)
  addsquare(y)
  return sumsquares.total

Of course this attribute belongs to the function (defintion), and not to the function call. So one must be mindful of threading and recursion.

Hemia

for python 2, provided the disclaimer above is paid attention to, this should be the accepted answer. I found it even allows you to nest a signal handler which can manipulate the outer functions state - I wouldn't suggest that though - just a first step in refactoring some god-awful legacy code which had globals everywhere. Now to refactor again and do it the right way... – Cattle

While this might be great for refactoring horrible global laden code, if you're writing your own and considering this, you should likely make the thing a class. But, this is certainly an improvement over randomish globals. And should be preferred with regard to some some static values. – Fanfaronade

Answer 5

This is a variation of redman's solution, but using a proper namespace instead of an array to encapsulate the variable:

def foo():
    class local:
        counter = 0
    def bar():
        print(local.counter)
        local.counter += 1
    bar()
    bar()
    bar()

foo()
foo()

I'm not sure if using a class object this way is considered an ugly hack or a proper coding technique in the python community, but it works fine in python 2.x and 3.x (tested with 2.7.3 and 3.2.3). I'm also unsure about the run-time efficiency of this solution.

Branch

Handy when local method has a lot of local variables to modify but me too I wonder " if using a class object this way is considered an ugly hack or a proper coding technique in the python community " – Twocolor

Answer 6

More from a philosophical point of view, one answer might be "if you're having namespace problems, give it a namespace of its very own!"

Providing it in its own class not only allows you to encapsulate the problem but also makes testing easier, eliminates those pesky globals, and reduces the need to shovel variables around between various top-level functions (doubtless there'll be more than just get_order_total).

Preserving the OP's code to focus on the essential change,

class Order(object):
  PRICE_RANGES = {
                  64:(25, 0.35),
                  32:(13, 0.40),
                  16:(7, 0.45),
                  8:(4, 0.5)
                  }


  def __init__(self):
    self._total = None

  def get_order_total(self, quantity):
      self._total = 0
      _i = self.PRICE_RANGES.iterkeys()
      def recurse(_i):
          try:
              key = _i.next()
              if quantity % key != quantity:
                  self._total += self.PRICE_RANGES[key][0]
              return recurse(_i) 
          except StopIteration:
              return (key, quantity % key)

      res = recurse(_i)

#order = Order()
#order.get_order_total(100)

As a PS, one hack which is a variant on the list idea in another answer, but perhaps clearer,

def outer():
  order = {'total': 0}

  def inner():
    order['total'] += 42

  inner()

  return order['total']

print outer()

Islean

Answer 7

While I used to use @redman's list-based approach, it's not optimal in terms of readability.

Here is a modified @Hans' approach, except I use an attribute of the inner function, rather than the outer. This should be more compatible with recursion, and maybe even multithreading:

def outer(recurse=2):
    if 0 == recurse:
        return

    def inner():
        inner.attribute += 1

    inner.attribute = 0
    inner()
    inner()
    outer(recurse-1)
    inner()
    print "inner.attribute =", inner.attribute

outer()
outer()

This prints:

inner.attribute = 3
inner.attribute = 3
inner.attribute = 3
inner.attribute = 3

If I s/inner.attribute/outer.attribute/g, we get:

outer.attribute = 3
outer.attribute = 4
outer.attribute = 3
outer.attribute = 4

So, indeed, it seems better to make them the inner function's attributes.

Also, it seems sensible in terms of readability: because then the variable conceptually relates to the inner function, and this notation reminds the reader that the variable is shared between the scopes of the inner and the outer functions. A slight downside for the readability is that the inner.attribute may only be set syntactically after the def inner(): ....

Dryclean

Answer 8

My way around...

def outer():

class Cont(object):
    var1 = None
    @classmethod
    def inner(cls, arg):
        cls.var1 = arg


Cont.var1 = "Before"
print Cont.var1
Cont.inner("After")
print Cont.var1

outer()

Nightwear

Answer 9

>>> def get_order_total(quantity):
    global PRICE_RANGES

    total = 0
    _i = PRICE_RANGES.iterkeys()
    def recurse(_i):
    print locals()
    print globals()
        try:
            key = _i.next()
            if quantity % key != quantity:
                total += PRICE_RANGES[key][0]
            return recurse(_i)
        except StopIteration:
            return (key, quantity % key)
    print 'main function', locals(), globals()

    res = recurse(_i)


>>> get_order_total(20)
main function {'total': 0, 'recurse': <function recurse at 0xb76baed4>, '_i': <dictionary-keyiterator object at 0xb6473e64>, 'quantity': 20} {'__builtins__': <module '__builtin__' (built-in)>, 'PRICE_RANGES': {64: (25, 0.34999999999999998), 32: (13, 0.40000000000000002), 16: (7, 0.45000000000000001), 8: (4, 0.5)}, '__package__': None, 's': <function s at 0xb646adf4>, 'get_order_total': <function get_order_total at 0xb646ae64>, '__name__': '__main__', '__doc__': None}
{'recurse': <function recurse at 0xb76baed4>, '_i': <dictionary-keyiterator object at 0xb6473e64>, 'quantity': 20}
{'__builtins__': <module '__builtin__' (built-in)>, 'PRICE_RANGES': {64: (25, 0.34999999999999998), 32: (13, 0.40000000000000002), 16: (7, 0.45000000000000001), 8: (4, 0.5)}, '__package__': None, 's': <function s at 0xb646adf4>, 'get_order_total': <function get_order_total at 0xb646ae64>, '__name__': '__main__', '__doc__': None}
{'recurse': <function recurse at 0xb76baed4>, '_i': <dictionary-keyiterator object at 0xb6473e64>, 'quantity': 20}
{'__builtins__': <module '__builtin__' (built-in)>, 'PRICE_RANGES': {64: (25, 0.34999999999999998), 32: (13, 0.40000000000000002), 16: (7, 0.45000000000000001), 8: (4, 0.5)}, '__package__': None, 's': <function s at 0xb646adf4>, 'get_order_total': <function get_order_total at 0xb646ae64>, '__name__': '__main__', '__doc__': None}
{'recurse': <function recurse at 0xb76baed4>, '_i': <dictionary-keyiterator object at 0xb6473e64>, 'quantity': 20}
{'__builtins__': <module '__builtin__' (built-in)>, 'PRICE_RANGES': {64: (25, 0.34999999999999998), 32: (13, 0.40000000000000002), 16: (7, 0.45000000000000001), 8: (4, 0.5)}, '__package__': None, 's': <function s at 0xb646adf4>, 'get_order_total': <function get_order_total at 0xb646ae64>, '__name__': '__main__', '__doc__': None}

Traceback (most recent call last):
  File "<pyshell#32>", line 1, in <module>
    get_order_total(20)
  File "<pyshell#31>", line 18, in get_order_total
    res = recurse(_i)
  File "<pyshell#31>", line 13, in recurse
    return recurse(_i)
  File "<pyshell#31>", line 13, in recurse
    return recurse(_i)
  File "<pyshell#31>", line 12, in recurse
    total += PRICE_RANGES[key][0]
UnboundLocalError: local variable 'total' referenced before assignment
>>> 

as you see, total is in the local scope of the main function, but it's not in the local scope of recurse (obviously) but neither it is in the global scope, 'cause it's defined only in the local scope of get_order_total

Photoconduction