Why does the ipython magic function `%timeit -n1 code_block` execute `code_block` multiple times?

Question

I am trying to run a particular test multiple times in ipython using the %timeit magic function. For demonstration purposes, I will just use -n1 instead of -n3 here, and use a simple print(1) function.

The %%timeit and the %timeit help says the following:

Options: -n<N>: execute the given statement <N> times in a loop. If this
value is not given, a fitting value is chosen.

-r<R>: repeat the loop iteration <R> times and take the best result.
Default: 3 (the 3 here is a typo in ipython, for which I have submitted a
PR)

However, if I do the following:

%%timeit -n1
print(1)

or

%timeit -n1 print(1)

it actually prints 1 7 times in a row as follows

In[1]: %timeit -n1 print(1)
1
1
1
1
1
1
1
32.8 µs ± 38.7 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

I was expecting that because of the definition of %%timeit/%timeit it would run the cell or code just once.

I have read this post: https://mcmap.net/q/498961/-why-does-timeit-loop-different-number-of-times which gives some examples of how %%timeit runs and the actual source code of ipython magic function %%timeit here: https://github.com/ipython/ipython/blob/ec3d1a11bf26a0962cb2cf55ba263b12ac023183/IPython/core/magics/execution.py#L944

where they define 2 types of loops: 1) -n<N> and 2) -r<R>.

If I just use -n1, it seems that it also assumes that I have used -r7, i.e. -n1 defaults to -n1 -r7. This means that even though I want it to run for exactly 1 run, it will still run the code_block 7 times as per https://github.com/ipython/ipython/blob/ec3d1a11bf26a0962cb2cf55ba263b12ac023183/IPython/core/magics/execution.py#L1021 unless I also specify -n1 -r1.

Questions:

1. Why are there 2 different ways to running the code_block using -n<N> and -r<R>?
2. What is the difference between -n<N> and -r<R> and why is this necessary?

Chaetognath

%timeit -r1 time.sleep(2) – Gluttony

Answer 1

These parameters are also in the timeit module.

• -n determines how many times you run the function (or block, or whatever) inside the timing window. So the stopwatch starts, the code is run n times, then the stopwatch ends. You should run it enough times that the results are meaningful (timeit defaults to powers of 10 until 0.2 seconds have elapsed).
• -r determines how many of these repetitions (where repetition is "start timer, run n times, stop timer") you should do. There is always some error due to your CPU scheduling other processes, etc., so usually you want to run it a few times and the best value of these r times is taken. (timeit defaults to 3, and the comments in the source you link indicate that ipython does the same -- but the actual code may disagree).

In pseudo-python, you can see how n and r affect the timing process:

time_hist = []
for _ in range(r):
    t0 = time.now()              # Start stopwatch (.now() is not a real function)
    for _ in range(n):
        # <your code block>
    t1 = time.now()              # Stop stopwatch

    time_hist.append(t1 - t0)    # Append time delta

 return min(time_hist)           # Return the min of the r deltas   

Seto

Thanks for replying.. actually I did check the timeit module in python as well before posting.. and did see this.. What I was wondering is why it is necessary to have 2 different options -n and -r, why not just have 1? I will check the code that you mentioned in some more detail to see if there is any difference.. – Chaetognath

I played around with the function you mentioned.. A follow up question..When using just -n, the fundamental reason why you might get different values when running the same function/code, is because the underlying system may be behaving slightly differently from 1 run to another. To compensate for that we can use -n such that the total time is ~0.2s or higher. That makes sense.. This also takes care and accounts for the fact that CPU may be scheduling other processes, or there may be network connections or you need to access the hard-disk which may be in use by something else – Chaetognath

All these variabilities are accounted for by having a sufficiently large -n >=0.2s. As you mentioned -r determines how many repetitions of -n you are doing and its also taking care of the same variability (CPU scheduling, Network communications, hard-disk access etc). So if we used a sufficiently large -r, with -n1, wouldn't it just be doing the same thing? Because, fundamentally we are trying to make sure that we have sampled enough times, so that the variability in process time due to CPU scheduling, network access, hard disk access are accounted for.. – Chaetognath

Because regardless of whether you use -n20 -r1, or -n1 -r20, or just had 1 option which just provided an option to define the number of times the function is run, say -l20, they are all just doing random sampling, and capturing sufficient number of samples to have high signal/noise ratio? – Chaetognath