Django test to use existing database

Question

I'm having a hard time customizing the test database setup behavior. I would like to achieve the following:

• The test suites need to use an existing database
• The test suite shouldn't erase or recreate the database instead load the data from a mysql dump
• Since the db is populated from a dump, no fixtures should be loaded
• Upon finishing tests the database shouldn't be destroyed

I'm having a hard time getting the testsuiterunner to bypass creation.

Doriandoric

In what way are you using the existing database if you are loading data from an sql dump? I would suggest loading the dump, creating fixtures, and using the conventional testing approach. – Goby

there is a lot of data and loading from dump is faster than loading from fixtures – Doriandoric

So it is not about using the existing database. Are you 100% sure that you need to do the testing using the complete data? Can a sample be sufficient for testing? – Goby

not "the" existing database,but "an" existing database that contains part of the data. – Doriandoric

Tests are design to build and subsequently tear down a database for a reason. They're not supposed to be interacting with data that has any persistence. I think you'll find it very difficult to change this behavior if it's even possible. You best bet is to use fixtures or a sqldump as the others suggested. – Dictatorship

Disappointing that no one has an answer for this. We have 6 GB read-only database with a ton of lookup tables that would need to be in place for tests to work... (Most of the data copied from elsewhere, but it would be oh so much more efficient to just use the existing db, instead of extracting the lookups or just dumping the entire thing) – Curvy

Hey, if you also got here from Google like me, see here: #4607256 – Curvy

@Curvy starting a bounty will make the question more visible to other community members. – Bend

The docs say: "If you require data for a test case, you should add it using either a test fixture, or programmatically add it during the setUp() of your test case." docs.djangoproject.com/en/dev/howto/initial-data/… – Plebe

possible duplicate of How to run django unit-tests on production database? – Birthroot

Duplicate of both stackoverflow.com/questions/1646468 and already mentioned stackoverflow.com/questions/4606756 – Birthroot

Answer 1

Fast forward to 2016 and the ability to retain the database between tests has been built into django. It's available in the form of the --keep flag to manage.py

New in Django 1.8. Preserves the test database between test runs. This has the advantage of skipping both the create and destroy actions which can greatly decrease the time to run tests, especially those in a large test suite. If the test database does not exist, it will be created on the first run and then preserved for each subsequent run. Any unapplied migrations will also be applied to the test database before running the test suite.

This pretty much fullfills all the criteria you have mentioned in your questions. In fact it even goes one step further. There is no need to import the dump before each and every run.

Anticathexis

Above link(--keep flag) is broken now docs.djangoproject.com/en/3.1/ref/django-admin/… – Proselytize

Answer 2

This TEST_RUNNER works in Django 1.3

from django.test.simple import DjangoTestSuiteRunner as TestRunner

class DjangoTestSuiteRunner(TestRunner):
    def setup_databases(self, **kwargs):
        pass

    def teardown_databases(self, old_config, **kwargs):
        pass

Groundless

Answer 3

You'll need to provide a custom test runner.

The bits your interested in overriding with the default django.test.runner.DiscoverRunner are the DiscoverRunner.setup_databases and DiscoverRunner.teardown_databases methods. These two methods are involved with creating and destroying test databases and are executed only once. You'll want to provide test-specific project settings that use your existing test database by default and override these so that the dump data is loaded and the test database isn't destroyed.

Depending on the size and contents of the dump, a safe bet might be to just create a subprocess that will pipe the dump to your database's SQL command-line interface, otherwise you might be able to obtain a cursor and execute queries directly.

If your looking to get rid of fixture loading completely, you can provide a custom base test case that extends Django's default django.test.testcases.TestCase with the TestCase._fixutre_setup and TestCase._fixutre_teardown methods overriden to be noop.

Caveat emptor: this runner will make it impossible to facilitate tests for anything but your application's sources. It's possible to customize the runner to create a specific alias for a connection to your existing database and load the dump, then provide a custom test case that overrides TestCase._database_names to point to it's alias.

Wimsatt