django unit tests without a db

Question

Is there a possibility to write django unittests without setting up a db? I want to test business logic which doesn't require the db to set up. And while it is fast to setup a db, I really don't need it in some situations.

Provincetown

I am wondering if that actually matters. The db is kept in memory + if you don't have any models nothing is performed with the db. So if you don't need it don't set up models. – Fillagree

I do have models, but for those tests they are not relevant. And the db is not kept in the memory, but built up in mysql, however, specifically for this purpose. Not that I want this.. Maybe I could configure django to use an in-memory db for testing. Do you know how to do this? – Provincetown

Oh, I am sorry. In-memory databases are just the case when you use an SQLite database. Except this I don't see a way to avoid creating the test db. There is nothing about this in the docs + I never felt the need to avoid it. – Fillagree

The accepted answer didn't work me. Instead, this worked perfectly: caktusgroup.com/blog/2013/10/02/skipping-test-db-creation – Gilles

As of Django 3.1, there is also a MIGRATE database setting: "When set to False, migrations won’t run when creating the test database. This is similar to setting None as a value in MIGRATION_MODULES, but for all apps." – Rockbound

Answer 1

You can subclass DjangoTestSuiteRunner and override setup_databases and teardown_databases methods to pass.

Create a new settings file and set TEST_RUNNER to the new class you just created. Then when you're running your test, specify your new settings file with --settings flag.

Here is what I did:

Create a custom test suit runner similar to this:

from django.test.simple import DjangoTestSuiteRunner

class NoDbTestRunner(DjangoTestSuiteRunner):
  """ A test runner to test without database creation """

  def setup_databases(self, **kwargs):
    """ Override the database creation defined in parent class """
    pass

  def teardown_databases(self, old_config, **kwargs):
    """ Override the database teardown defined in parent class """
    pass

Create a custom settings:

from mysite.settings import *

# Test runner with no database creation
TEST_RUNNER = 'mysite.scripts.testrunner.NoDbTestRunner'

When you're running your tests, run it like the following with --settings flag set to your new settings file:

python manage.py test myapp --settings='no_db_settings'

UPDATE: April/2018

Since Django 1.8, the module django.test.simple.DjangoTestSuiteRunner were moved to 'django.test.runner.DiscoverRunner'.

For more info check official doc section about custom test runners.

Zampardi

looking forward to your exaple! – Provincetown

Just added the example. Feel free to ask if there's something you don't understand about it. – Zampardi

Now it complains about a KeyError: 'SUPPORTS_TRANSACTIONS'. Did you try to run your solution? – Provincetown

This error raises when you have tests that need database transactions. Obviously if you don't have a DB, you're not going to be able to run those tests. You should run your tests separately. If you just run your test by using python manage.py test --settings=new_settings.py, it is going to run a whole bunch of other tests from other apps that may require database. – Zampardi

Note that you'll need to extend SimpleTestCase instead of TestCase for your test classes. TestCase expects a database. – Fulllength

If you don't want to use a new settings file you can specify the new TestRunner on the command line with the --testrunner option. – Hardball

Great answer!! In django 1.8, from django.test.simple import DjangoTestSuiteRunner has been changed to from django.test.runner import DiscoverRunner Hope that helps someone! – Barksdale

Ben Roberts comments worked for me just fine! I removed all the django dependencies that uses DB (namely "django.contrib.auth", "django.contrib.contenttypes" and "django.contrib.sites"), made DATABASES = {} on my settings.py file and inherited my tests from SimpleTestCase instead of TestCase – Severance

This doesn't work in Django 1.10. Also, it is a misleading answer, as deriving from SimpleTestCase – Countercheck

runtests is broken in Django 1.8 github.com/chrisdev/django-pandas/issues/45 – Threshold

In Django 1.8 and above, a slight correction to the above code can be made. The import statement can be changed to : from django.test.runner import DiscoverRunner The NoDbTestRunner must now extend the DiscoverRunner class. – Diorite

As of django 2.2, doing it this way results in an error: TypeError: Model instances without primary key value are unhashable – Equipoise

For django==2.2.13, I derive my test class from SimpleTestCase. I use either --testrunner argument or settings for targeting NoDbTestRunner runner. Still tests fail if I point "default" DB connection to non-existing address. So... does NoDbTestRunner hack really allow running test WITHOUT any database (and that means I did somethingwrong) or a DB is still needed just to run the test? – Erleena

Answer 2

Generally tests in an application can be classified in to two categories

1. Unit tests, these test the individual snippets of code in insolation and do not require to go to the database
2. Integration test cases which actually go to the database and test the fully integrated logic.

Django supports both unit and integration tests.

Unit tests, do not require to setup and tear down database and these we should inherit from SimpleTestCase.

from django.test import SimpleTestCase


class ExampleUnitTest(SimpleTestCase):
    def test_something_works(self):
        self.assertTrue(True)

For integration test cases inherit from TestCase in turn inherits from TransactionTestCase and it will setup and tear down the database before running each test.

from django.test import TestCase


class ExampleIntegrationTest(TestCase):
    def test_something_works(self):
        #do something with database
        self.assertTrue(True)

This strategy will ensure that database in created and destroyed only for the test cases that access the database and therefore tests will be more efficient

Glassy

This might make running tests more efficient, but note that the test runner still creates test databases on initialization. – Heinrike

So much simpler that the chosen answer. Thank you so much! – Graybeard

@Heinrike No... if u have only SimpleTestCase class, the test runner don't run anything, see this project. – Kalfas

Django will still try to create a test DB even if you only use SimpleTestCase. See this question. – Fieldwork

using SimpleTestCase exactly works for testing utility methods or snippets and doesn't use or create test db. Exactly what I need! – Algonquian

Answer 3

From django.test.simple

  warnings.warn(
      "The django.test.simple module and DjangoTestSuiteRunner are deprecated; "
      "use django.test.runner.DiscoverRunner instead.",
      RemovedInDjango18Warning)

So override DiscoverRunner instead of DjangoTestSuiteRunner.

 from django.test.runner import DiscoverRunner
 
 class NoDbTestRunner(DiscoverRunner):
   """ A test runner to test without database creation/deletion """
 
   def setup_databases(self, **kwargs):
     pass
      
   def teardown_databases(self, old_config, **kwargs):
     pass

Use like that :

python manage.py test --testrunner=app.filename.NoDbTestRunner app

Deli

Answer 4

I chose to inherit from django.test.runner.DiscoverRunner and make a couple of additions to the run_tests method.

My first addition checks to see if setting up a db is necessary and allows the normal setup_databases functionality to kick in if a db is necessary. My second addition allows the normal teardown_databases to run if the setup_databases method was allowed to run.

My code assumes that any TestCase that inherits from django.test.TransactionTestCase (and thus django.test.TestCase) requires a database to be setup. I made this assumption because the Django docs say:

If you need any of the other more complex and heavyweight Django-specific features like ... Testing or using the ORM ... then you should use TransactionTestCase or TestCase instead.

https://docs.djangoproject.com/en/1.6/topics/testing/tools/#django.test.SimpleTestCase

mysite/scripts/settings.py

from django.test import TransactionTestCase     
from django.test.runner import DiscoverRunner


class MyDiscoverRunner(DiscoverRunner):
    def run_tests(self, test_labels, extra_tests=None, **kwargs):
        """
        Run the unit tests for all the test labels in the provided list.

        Test labels should be dotted Python paths to test modules, test
        classes, or test methods.

        A list of 'extra' tests may also be provided; these tests
        will be added to the test suite.

        If any of the tests in the test suite inherit from
        ``django.test.TransactionTestCase``, databases will be setup. 
        Otherwise, databases will not be set up.

        Returns the number of tests that failed.
        """
        self.setup_test_environment()
        suite = self.build_suite(test_labels, extra_tests)
        # ----------------- First Addition --------------
        need_databases = any(isinstance(test_case, TransactionTestCase) 
                             for test_case in suite)
        old_config = None
        if need_databases:
        # --------------- End First Addition ------------
            old_config = self.setup_databases()
        result = self.run_suite(suite)
        # ----------------- Second Addition -------------
        if need_databases:
        # --------------- End Second Addition -----------
            self.teardown_databases(old_config)
        self.teardown_test_environment()
        return self.suite_result(suite, result)

Finally, I added the following line to my project's settings.py file.

mysite/settings.py

TEST_RUNNER = 'mysite.scripts.settings.MyDiscoverRunner'

Now, when running only non-db-dependent tests, my test suite runs an order of magnitude faster! :)

Natiha

Answer 5

Updated: also see this answer for using a third-party tool pytest.

---

@Cesar is right. After accidentally running ./manage.py test --settings=no_db_settings, without specifying an app name, my development database was wiped out.

For a safer manner, use the same NoDbTestRunner, but in conjunction with the following mysite/no_db_settings.py:

from mysite.settings import *

# Test runner with no database creation
TEST_RUNNER = 'mysite.scripts.testrunner.NoDbTestRunner'

# Use an alternative database as a safeguard against accidents
DATABASES['default']['NAME'] = '_test_mysite_db'

You need to create a database called _test_mysite_db using an external database tool. Then run the following command to create the corresponding tables:

./manage.py syncdb --settings=mysite.no_db_settings

If you're using South, also run the following command:

./manage.py migrate --settings=mysite.no_db_settings

OK!

You can now run unit tests blazingly fast (and safe) by:

./manage.py test myapp --settings=mysite.no_db_settings

Recuperator

I have run tests using pytest (with pytest-django plugin) and NoDbTestRunner, if somehow you create an object by accident in a testcase and you don't override database name, the object will be created in your local databases that you setup in the settings. The name 'NoDbTestRunner' should be 'NoTestDbTestRunner' because it won't create the test database, but will use your database from settings. – Lannylanolin

Answer 6

As an alternative to modifying your settings to make NoDbTestRunner "safe", here's a modified version of NoDbTestRunner that closes the current database connection and removes the connection information from settings and the connection object. Works for me, test it in your environment before relying on it :)

class NoDbTestRunner(DjangoTestSuiteRunner):
    """ A test runner to test without database creation """

    def __init__(self, *args, **kwargs):
        # hide/disconnect databases to prevent tests that 
        # *do* require a database which accidentally get 
        # run from altering your data
        from django.db import connections
        from django.conf import settings
        connections.databases = settings.DATABASES = {}
        connections._connections['default'].close()
        del connections._connections['default']
        super(NoDbTestRunner,self).__init__(*args,**kwargs)

    def setup_databases(self, **kwargs):
        """ Override the database creation defined in parent class """
        pass

    def teardown_databases(self, old_config, **kwargs):
        """ Override the database teardown defined in parent class """
        pass

Wiretap

NOTE: If you delete the default connection from the connections list you won't be able to use Django models or other features that normally use the database (obviously we don't communicate with the database but Django checks different features that the DB supports). Also it seems that connections._connections does not support __getitem__ anymore. Use connections._connections.default in order to access the object. – Accountable

Answer 7

Another solution would be to have your test class simply inherit from unittest.TestCase instead of any of Django's test classes. The Django docs (https://docs.djangoproject.com/en/2.0/topics/testing/overview/#writing-tests) contain the following warning about this:

Using unittest.TestCase avoids the cost of running each test in a transaction and flushing the database, but if your tests interact with the database their behavior will vary based on the order that the test runner executes them. This can lead to unit tests that pass when run in isolation but fail when run in a suite.

However, if your test doesn't use the database, this warning needn't concern you and you can reap the benefits of not having to run each test case in a transaction.

Meditation

It looks like this still creates and destroys the db, the only difference is that it doesn't run the test in a transaction and doesn't flush the db. – Thalassa

Answer 8

The above solutions are fine too. But the following solution will also reduce the db creation time if there are more number of migrations. During unit testing, running syncdb instead of running all the south migrations will be much faster.

SOUTH_TESTS_MIGRATE = False # To disable migrations and use syncdb instead

Shaddock

Answer 9

My web host only allows creating and dropping databases from their Web GUI, so I was getting a "Got an error creating the test database: Permission denied" error when trying to run python manage.py test.

I'd hoped to use the --keepdb option to django-admin.py but it doesn't seem to be supported any longer as of Django 1.7.

What I ended up doing was modifying the Django code in .../django/db/backends/creation.py, specifically the _create_test_db and _destroy_test_db functions.

For _create_test_db I commented out the cursor.execute("CREATE DATABASE ... line and replaced it with pass so the try block wouldn't be empty.

For _destroy_test_db I just commented out cursor.execute("DROP DATABASE - I didn't need to replace it with anything because there was already another command in the block (time.sleep(1)).

After that my tests ran fine - though I did set up a test_ version of my regular database separately.

This isn't a great solution of course, because it will break if Django is upgraded, but I had a local copy of Django due to using virtualenv so at least I have control over when/if I upgrade to a newer version.

Ansilma

Answer 10

Another solution not mentioned: this was easy for me to implement because I already have multiple settings files (for local / staging / production) that inherit from base.py . So unlike other people I did not have to overwrite DATABASES['default'], as DATABASES isn't set in base.py

SimpleTestCase still tried to connect to my test database and run migrations. When I made a config/settings/test.py file that didn't set DATABASES to anything, then my unit tests ran without it. It allowed me to use models that had foreign key and unique constraint fields. (Reverse foreign key lookup, which requires a db lookup, fails.)

(Django 2.0.6)

PS code snippets

PROJECT_ROOT_DIR/config/settings/test.py:
from .base import *
#other test settings

#DATABASES = {
# 'default': {
#   'ENGINE': 'django.db.backends.sqlite3',
#   'NAME': 'PROJECT_ROOT_DIR/db.sqlite3',
# }
#}

cli, run from PROJECT_ROOT_DIR:
./manage.py test path.to.app.test --settings config.settings.test

path/to/app/test.py:
from django.test import SimpleTestCase
from .models import *
#^assume models.py imports User and defines Classified and UpgradePrice

class TestCaseWorkingTest(SimpleTestCase):
  def test_case_working(self):
    self.assertTrue(True)
  def test_models_ok(self):
    obj = UpgradePrice(title='test',price=1.00)
    self.assertEqual(obj.title,'test')
  def test_more_complex_model(self):
    user = User(username='testuser',email='[email protected]')
    self.assertEqual(user.username,'testuser')
  def test_foreign_key(self):
    user = User(username='testuser',email='[email protected]')
    ad = Classified(user=user,headline='headline',body='body')
    self.assertEqual(ad.user.username,'testuser')
  #fails with error:
  def test_reverse_foreign_key(self):
    user = User(username='testuser',email='[email protected]')
    ad = Classified(user=user,headline='headline',body='body')
    print(user.classified_set.first())
    self.assertTrue(True) #throws exception and never gets here

Empyema

Answer 11

When using the nose test runner (django-nose), you can do something like this:

my_project/lib/nodb_test_runner.py:

from django_nose import NoseTestSuiteRunner


class NoDbTestRunner(NoseTestSuiteRunner):
    """
    A test runner to test without database creation/deletion
    Used for integration tests
    """
    def setup_databases(self, **kwargs):
        pass

    def teardown_databases(self, old_config, **kwargs):
        pass

In your settings.py you can specify the test runner there, i.e.

TEST_RUNNER = 'lib.nodb_test_runner.NoDbTestRunner' . # Was 'django_nose.NoseTestSuiteRunner'

OR

I wanted it for running specific tests only, so I run it like so:

python manage.py test integration_tests/integration_*  --noinput --testrunner=lib.nodb_test_runner.NoDbTestRunner

Equipoise

Answer 12

You can set databases to an empty list inside the normal TestCase from django.test.

from django.test import TestCase

class NoDbTestCase(TestCase):
    databases = []

Burleigh