Function that returns an accumulator in Python

Question

I am reading Hackers and Painters and am confused by a problem mentioned by the author to illustrate the power of different programming languages.

The problem is:

We want to write a function that generates accumulators—a function that takes a number n, and returns a function that takes another number i and returns n incremented by i. (That’s incremented by, not plus. An accumulator has to accumulate.)

The author mentions several solutions with different programming languages. For example, Common Lisp:

(defun foo (n)
  (lambda (i) (incf n i)))

and JavaScript:

function foo(n) { return function (i) { return n += i } }

However, when it comes to Python, the following codes do not work:

def foo(n):
    s = n
    def bar(i):
        s += i
        return s
    return bar

f = foo(0)
f(1)  # UnboundLocalError: local variable 's' referenced before assignment

A simple modification will make it work:

def foo(n):
    s = [n]
    def bar(i):
        s[0] += i
        return s[0]
    return bar

I am new to Python. Why doesn the first solution not work while the second one does? The author mentions lexical variables but I still don't get it.

Subchloride

You can read nested function variable scoping. – Lheureux

A def introduces a new scope. You cannot re-bind a variable in an outer scope without global or nonlocal. – Dietetics

Answer 1

s += i is just sugar for s = s + i.^*

This means you assign a new value to the variable s (instead of mutating it in place). When you assign to a variable, Python assumes it is local to the function. However, before assigning it needs to evaluate s + i, but s is local and still unassigned -> Error.

In the second case s[0] += i you never assign to s directly, but only ever access an item from s. So Python can clearly see that it is not a local variable and goes looking for it in the outer scope.

Finally, a nicer alternative (in Python 3) is to explicitly tell it that s is not a local variable:

def foo(n):
    s = n
    def bar(i):
        nonlocal s
        s += i
        return s
    return bar

(There is actually no need for s - you could simply use n instead inside bar.)

*_{The situation is slightly more complex, but the important issue is that computation and assignment are performed in two separate steps.}

Dative

s += i is just sugar for s = s + i — note that this is not true for some types. Using += on a list will absolutely mutate it (via __iadd__). – Dietetics

Thanks, now I understand. I use a redundant s just to make the comparison between the two cases clearer. BTW, thanks for the corrections of my poor grammar. :) – Subchloride

@Dietetics that is correct, but it makes the explanation easier :) The scoping rules also apply for these types, though. – Dative

"you could simply use n instead inside bar" by writing n = n + i? will Python not make a new, local n for it, just the same? – Leeleeann

@WillNess Sure. Function arguments are just like variables, so you would have to declare it as nonlocal inside bar. – Dative

Answer 2

An infinite generator is one implementation. You can call __next__ on a generator instance to extract successive results iteratively.

def incrementer(n, i):
    while True:
        n += i
        yield n

g = incrementer(2, 5)

print(g.__next__())  # 7
print(g.__next__())  # 12
print(g.__next__())  # 17

If you need a flexible incrementer, one possibility is an object-oriented approach:

class Inc(object):
    def __init__(self, n=0):
        self.n = n
    def incrementer(self, i):
        self.n += i
        return self.n

g = Inc(2)

g.incrementer(5)  # 7
g.incrementer(3)  # 10
g.incrementer(7)  # 17

Pizzeria

Yes, but it's not flexible since the increase is fixed to 5. – Subchloride

Thanks, that's correct, though more verbose than the one using closure. :) – Subchloride

Sometimes, verbose is more pythonic :). But this is more a question of style / preference. I personally think it's cleaner to use classes instead of global / nonlocal statements. – Pizzeria

Answer 3

In Python if we use a variable and pass it to a function then it will be Call by Value whatever changes you make to the variable it will not be reflected to the original variable.

But when you use a list instead of a variable then the changes that you make to the list in the functions are reflected in the original List outside the function so this is called call by reference.

And this is the reason for the second option does work and the first option doesn't.

Mcneil