ipython %timeit "local variable 'a' referenced before assignment"

Question

I am trying to the run the following code but I get a local variable 'a' referenced before assignment.

a = [x for x in range(10)]
b = [x for x in range(10)]
%timeit a+=b

The statement works without the %timeit magic.

Is there something I am missing?

Thank you.

Genetic

have you tried %timeit $a+=$b – Camaraderie

Thank you for taking the time to answer but that does not work. Neither surrounding the expression with braces. I thought the "$" and {braces} are only used when making system calls! – Genetic

yeah, sorry, too quick. Reopening. – Camaraderie

Answer 1

What do you expect it to do?

Outside of the timeit it does:

In [188]: a += b
In [189]: a
Out[189]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

I tried initializing x, and got a near infinite loop, eventually ending with a memory error

In [192]: %%timeit x=a
     ...: x += b

In [194]: len(a)
Out[194]: 529076630

In other words each timeit loop concatenated another list of b values to x (and by extension a), resulting are very long loop. I suspect an individual x+=b was fast, resulting in timeit choosing to loop many times.

Lets create an a fresh each loop:

In [196]: %%timeit
     ...: a = [x for x in range(10)]
     ...: a += b
     ...: 
1.91 µs ± 4.82 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

This produces the memory error as well:

In [197]: %%timeit a = [x for x in range(10)]
     ...: a += b

If I control the number of loops:

In [202]: %%timeit -n 100 a = [x for x in range(10)]
     ...: a += b
     ...: 
     ...: 
208 ns ± 11 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)

With ns times I can see why the default loops is so large.

I haven't tried timing a plain a+=... before (not even with numpy arrays), but evidently it expects some sort of local initialization for that a, either within the loop or in the initialization block. But it is important to keep in mind that the timed actions may be performed many times (the -r and -n parameters or the default values). So any in-place action might result bit changes to the global values. In this case timeit might be trying to protect us from that kind of unexpected growth, by expecting some sort of 'local' variable.

---

Lets try the a+b, but with an assignment:

In [215]: c=np.zeros(10)
In [216]: a
Out[216]: array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10])
In [217]: b
Out[217]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
In [218]: %timeit c = a+b
5.33 µs ± 105 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [219]: c
Out[219]: array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.])

Notice that the global c has not changed. The assignment is to a temporary local c - even though a global of the same name is available.

As a general rule, calculations performed inside the timing loop should not leak outside the loop. You have to something explicit as I did in the memory error loop, or here

In [222]: %%timeit x = c
     ...: x += b
     ...: 
9.04 µs ± 238 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [223]: c
Out[223]: 
array([       0.,   811111.,  1622222.,  2433333.,  3244444.,  4055555.,
        4866666.,  5677777.,  6488888.,  7299999.])

or here:

In [224]: c=np.zeros(10)
In [225]: %%timeit x = c
     ...: x[:] = a+b

7.84 µs ± 199 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [226]: c
Out[226]: array([  1.,   3.,   5.,   7.,   9.,  11.,  13.,  15.,  17.,  19.])

Both use an in-place assignment to a local variable which has been linked to a mutable global.

Barefaced

I just expect it to time a+=b. The following commands work: %timeit a+b %timeit a.extend(b) but: %timeit a+=b returns the error message that local variable 'a' referenced before assignment. I just want to understand why a is not known in the latter but works with the former. – Genetic

Which a is it supposed to use during each loop? The original 10 element list? Or the a produced by the last iteration (which keeps growing)? Or a new empty list? Do you understand why I got a memory error? – Barefaced

Details of how the expression is compiled, and how the user namespace is passed to it (along with a local namespace) are buried in timeit code (not just the magic, but also the underlying timeit module. docs.python.org/3.5/library/timeit.html – Barefaced