Can I run line_profiler over a pytest test?

Question

I have identified some long running pytest tests with

py.test --durations=10

I would like to instrument one of those tests now with something like line_profiler or cprofile. I really want to get the profile data from the test itself as the pytest setup or tear down could well be part of what is slow.

However given how line_profiler or cprofile is typically involved it isn't clear to me how to make them work with pytest.

Bridlewise

Answer 1

Run pytest like this:

python3 -m cProfile -o profile -m pytest

You can even pass in optional arguments:

python3 -m cProfile -o profile -m pytest tests/worker/test_tasks.py -s campaigns

This will create a binary file called profile in your current directory. This can be analyzed with pstats:

import pstats
p = pstats.Stats('profile')
p.strip_dirs()
p.sort_stats('cumtime')
p.print_stats(50)

This will print the 50 lines with the longest cumulative duration.

Leilanileininger

I've tried to get this working on Windows. But if fails on the first command, saying: "no such file of directory '(which'". I tried to give the absolute path to the py.test.exe binary but I get a different error: "TypeError: compile() expected string without null bytes". I got the profiling running by adding a pytest.main call in the test module. Any suggestions how to get it running from the command line on Windows? – Labret

From command line: python -c "import pstats; pstats.Stats('profile').strip_dirs().sort_stats('cumtime').print_stats(50)" – Adrienadriena

python -m cProfile -o profile -m pytest gives cProfile.py: error: no such option: -m. Try python3 -m cProfile -o profile $(which py.test) instead. – Cunningham

Answer 2

To get cProfile and line_profiler to work with py.test code, I did two things:

1. Extended the py.test test code with a call to pytest.main(), which made it executable with the python interpreter as the main driver:

   # pytest_test.py:
   @profile # for line_profiler only
   def test_example():
       x = 3**32
       assert x == 1853020188851841
   
   # for profiling with cProfile and line_profiler
   import pytest
   pytest.main(__file__)
   

   Now you can run this test without py.test as the main driver using other tools:

   $ kernprof.py -l pytest_test.py
   $ python -m line_profiler pytest_test.py.lprof
   

   or

   $ python -m cProfile pytest_test.py
   

2. To profile py.test-specific functions such as pytest_funcarg*() with line_profiler I split them in two to avoid confusion between py.test and line_profiler:

   def pytest_funcarg__foo(request):
       return foo(request)
   
   @profile
   def foo(request):
   ...
   

The same method works for memory_profiler.

Deflagrate

Answer 3

Have you tried the pytest-profiling plugin?

Cathryncathy

This is a really nice plugin which makes profiling really simple, but AFAIK it does not do line profiling – Differentia

Answer 4

I recently wrote pytest-line-profiler that may be useful. It's not battle tested, so please give me feedback (and help).

Presbyterate

It breaks some of my tests even when not used, just installed. – Omland

could you open an issue with an example? thanks! – Presbyterate

Answer 5

This also works with pyinstrument:

pyinstrument -m pytest test.py

Changsha