Trying to understand python memory profiler

Question

I am using Memory Profiler module to get the memory usage of my python code following this answer. However, I am not able to interpret the output from %memit magic (or the output using the @profile decorator from the module or mprof run for that matter).

For instance,

%memit range(10000) 

gives me peak memory: 97.41 MiB, increment: 0.24 MiB

while,

%memit xrange(10000)

shows peak memory: 97.46 MiB, increment: 0.00 MiB. I do understand the difference between xrange returning an xrange type as opposed to range() returning a list. I used them here just to demonstrate the two scenarios.

My question is

1. What does peak memory and increment actually mean?
2. What should I report as the total memory usage of a script (or a function) from this output?

Sibel

Answer 1

Peak memory refers to the peak memory usage of your system (including memory usage of other processes) during the program runtime.

Increment is the increment in memory usage relative to the memory usage just before the program is run (i.e. increment = peak memory - starting memory).

So you'd report increment. Peak memory just helps you figure how close you are to using all your RAM during a program run.

If you refer to the line-by-line example in the usage section of the readme:

Line #    Mem usage  Increment   Line Contents
==============================================
     3                           @profile
     4      5.97 MB    0.00 MB   def my_func():
     5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)
     6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)
     7     13.61 MB -152.59 MB       del b
     8     13.61 MB    0.00 MB       return a

%memit is essentially giving you the memory usage from line 6, but reports the increment relative to line 4 (actually line 1, but presumably, there's no computation in lines 1-3).

Gildea