How to use cProfile with nosetest --with-profile?

Question

nosetest --with-profile --profile-stats-file output

The output can't read by runsnake, because nosetest uses hotshot, if I want to generate a file that can be read with runsnake, I need to convert it so:

st = hotshot.stats.load('output')

st.dump_stats('output_new')

Could I run the test with cProfile directly for read with runsnake?

Lemire

seems weird that nosetests would choose hotshot – Waddington

Maybe this will be handy: github.com/msherry/nose-cprof ? (I did not try this myself yet) – Alike

Answer 1

Evolving on the answer of @squid, you can use a nose plugin called nose-cprof to replace the nose default profiler, hotshot, with cProfile.

To install it:

pip install nose-cprof

Then call nose like this:

nosetests --with-cprofile

It should generate a cProfile output file, that you can then analyze with tools like runsnakerun.

Tracey

I've not had luck with getting the plugin to work: pip install looks fine, but when running --with-cprofile, the flags are not found. I went for the --with-profile instead. – Tevis

Answer 2

@cihanpesend's answer didn't quite work for me (cProfile couldn't find 'nosetests'), but I did have success on Linux using:

python -m cProfile -o profile.out `which nosetests` .

The resulting output works fine in runsnake.

(Presumably on Windows you could replace which nosetests with the hard-coded path to your nosetests top-level python script.)

I think you are right that the output from nosetests' hotshot profiler is not compatible with runsnake. Certainly the two don't play nice together out of the box for me either.

Me

Answer 3

I don't have info about nosetest except it is python project. So;

python -m cProfile -o outputfile nosetest

Then,

runsnake outputfile

RunSnakeRun is extremly useful to visualize profiler.

Note: to run runsnake, you must install wx and numpy.

update: from omikron's comment; runsnakerun can not support python3 profile output. (i didn't try)

Arraign

It should be noted that runsnakerun doesn't work when profiling Python 3 code and seem to be not maintained anymore. Instead I recommend to use pyprof2calltree with kcachegrind. – Maugham

Answer 4

Or you can try nose-cprof plugin:　https://github.com/msherry/nose-cprof　

It replaces hotshot with cProfile

Picro

Answer 5

With pyprof2calltree:

$ pip install pyprof2calltree
$ nosetests --with-cprofile --profile-stats=profile.out tests/
$ pyprof2calltree -i profile.out -k

With xdot:

$ sudo apt install xdot
$ gprof2dot -f pstats profile.out | dot -Tpng -o profile.png

Panada