How to see the traceback for each Django query?

Question

I want a traceback from every query executed during a request, so I can find where they're coming from and reduce the count/complexity.

I'm using this excellent snippet of middleware to list and time queries, but I don't know where in the they're coming from.

I've poked around in django/db/models/sql/compiler.py but apparent form getting a local version of django and editing that code I can't see how to latch on to queries. Is there a signal I can use? it seems like there isn't a signal on every query.

Is it possible to specify the default Manager?

(I know about django-toolbar, I'm hoping there's a solution without using it.)

Merrygoround

What do you mean where they are coming from? The request or the code base? – Botticelli

Answer 1

An ugly but effective solution (eg. it prints the trace on all queries and only requires one edit) is to add the following to the bottom of settings.py:

import django.db.backends.utils as bakutils
import traceback

bakutils.CursorDebugWrapper_orig = bakutils.CursorWrapper

def print_stack_in_project():
    stack = traceback.extract_stack()
    for path, lineno, func, line in stack:
        if 'lib/python' in path or 'settings.py' in path:
            continue
        print 'File "%s", line %d, in %s' % (path, lineno, func)
        print '  %s' % line

class CursorDebugWrapperLoud(bakutils.CursorDebugWrapper_orig):
    def execute(self, sql, params=None):
        try:
            return super(CursorDebugWrapperLoud, self).execute(sql, params)
        finally:
            print_stack_in_project()
            print sql
            print '\n\n\n'

    def executemany(self, sql, param_list):
        try:
            return super(CursorDebugWrapperLoud, self).executemany(sql, param_list)
        finally:
            print_stack_in_project()
            print sql
            print '\n\n\n'

bakutils.CursorDebugWrapper = CursorDebugWrapperLoud

Still not sure if there is a more elegant way of doing this?

Merrygoround

Shouldn't that be bakutils.CursorDebugWrapper_orig = bakutils.CursorDebugWrapper? – Gardening

Thanks for the answer! This gist contains version for Python 3 gist.github.com/Sinkler/a3108648b3aa4f867681ec11ac1b4529 – Anus

Answer 2

Django debug toolbar will tell you what you want with spectacular awesomeness.

Vitrescence

Answer 3

You could customize Django's LOGGING setting like this to output a traceback on each logged query, along with the usual output of logging.StreamHandler:

import logging
import traceback


class TracebackHandler(logging.StreamHandler):
    def emit(self, record):
        traceback.print_stack()
        super().emit(record)


LOGGING = {
    "version": 1,
    "handlers": {
        "traceback": {"class": f"{__name__}.TracebackHandler"},
    },
    "loggers": {
        "django.db": {
            "handlers": ["traceback"],
            "level": "DEBUG",
        },
    },
}

Alvord