How can I analyze Python code to identify problematic areas? [closed]

Question

I have a large source repository split across multiple projects. I would like to produce a report about the health of the source code, identifying problem areas that need to be addressed.

Specifically, I'd like to call out routines with a high cyclomatic complexity, identify repetition, and perhaps run some lint-like static analysis to spot suspicious (and thus likely erroneous) constructs.

How might I go about constructing such a report?

Shrievalty

Answer 1

For measuring cyclomatic complexity, there's a nice tool available at traceback.org. The page also gives a good overview of how to interpret the results.

+1 for pylint. It is great at verifying adherence to coding standards (be it PEP8 or your own organization's variant), which can in the end help to reduce cyclomatic complexity.

Radke

traceback.org is broken link – Pembroke

Pylint is great, but slow. We use as a pre-push hook, but use flake8 and a bunch of plugins for regular checks and pre-commit hook. I'd actually recommend https://flakehell.readthedocs.io/config.html these days though, as it's execution model is much more robust and configurable. – Pharr

Answer 2

For cyclomatic complexity you can use radon: https://github.com/rubik/radon

(Use pip to install it: pip install radon)

Additionally it also has these features:

• raw metrics (these include SLOC, comment lines, blank lines, &c.)
• Halstead metrics (all of them)
• Maintainability Index (the one used in Visual Studio)

Frictional

What's the switch to get Halstead metrics? – Hussy

@qarma If I understand the docs, I don't think you can use the command line. You have to use the Python API. – Frictional

Answer 3

For static analysis there is pylint and pychecker. Personally I use pylint as it seems to be more comprehensive than pychecker.

For cyclomatic complexity you can try this perl program, or this article which introduces a python program to do the same

Restrained

Answer 4

Pycana works like charm when you need to understand a new project!

PyCAna (Python Code Analyzer) is a fancy name for a simple code analyzer for python that creates a class diagram after executing your code.

See how it works: http://pycana.sourceforge.net/

output:

Taub

Answer 5

Thanks to Pydev, you can integrate pylint in the Eclipse IDE really easily and get a code report each time you save a modified file.

Mcnew

Answer 6

Use flake8, which provides pep8, pyflakes, and cyclomatic complexity analysis in one tool

Gaona

Answer 7

There is a tool called CloneDigger that helps you find similar code snippets.

Xanthippe

It does not work with Python 3 and it never was well maintained. – Heady

Answer 8

For checking cyclomatic complexity, there is of course the mccabe package.

Installation:

$ pip install --upgrade mccabe

Usage:

$ python -m mccabe --min=6 path/to/myfile.py

Note the threshold of 6 above. Per this answer, scores >5 probably should be simplified.

Sample output with --min=3:

68:1: 'Fetcher.fetch' 3
48:1: 'Fetcher._read_dom_tag' 3
103:1: 'main' 3

It can optionally also be used via pylint-mccabe or pytest-mccabe, etc.

Heady