Django (nose) test speeding up with reuse_db not working

Question

I am using django-nose to run my unit tests in django (1.4).

TEST_RUNNER = 'django_nose.NoseTestSuiteRunner'

Creating the database takes a long time.

So I found out putting this in settings.py:

os.environ['REUSE_DB'] = "1"

should do the trick.

actually django itsellve gives this suggestion:

To reuse old database "<path not very interesting>/var/sqlite/unittest.db" for speed, set env var REUSE_DB=1.

of course you need to run it once (or after every database change) with this flag =0

However, when you set the flag to 0, my tests end with the remark:

Destroying test database for alias 'default'...

So when I want to run it with reuse.... there is nothing to reuse... and I will get errors saying the table does not exist

DatabaseError: no such table: <and than a table name>

The test runs perfectly when set the reuse_db to 0

I am using the test database alias in my development settings:

DATABASES = {
    'default': {
        'NAME': os.path.join(BUILDOUT_DIR, 'var', 'sqlite', 'development.db'),
        'TEST_NAME': os.path.join(BUILDOUT_DIR, 'var', 'sqlite', 'unittest.db'),
        'ENGINE': 'django.db.backends.sqlite3', 
        'USER': '',
        'PASSWORD': '',
        'HOST': '', 
        'PORT': '', 
        }
    }

I am not using the in-memory sqllite database for testing because I read somewhere this doesn't work well with django-nose.

So how can I reuse the DB when it is destroying the databse in the end...

according to this https://docs.djangoproject.com/en/1.4/topics/testing/#the-test-database django is doing this, but it does not show how to prevent this (if I can), or how to use the reuse_db option. should I use other settings?

Dominican

Answer 1

If I have understood correctly, you don't know how to create the test database first time (in order to reuse it later).

NoseTestSuiteRunner should create it automatically if DB does not exist even if you set REUSE_DB = 0. If you want to create test DB manually you can create the following file:

test_db_settings.py

in which you specify:

from settings import *

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        # TestRunner uses a database with name that consists
        # of prefix test_ and your database name if your database
        # name is db then the test database name has to be test_db
        'NAME': 'test_db', 
        'USER': 'postgres_user',
        'PASSWORD': 'postgres_user',
        'HOST': 'localhost',
        'PORT': '5432',
        }
}

after that create test_db:

createdb -U postgres_user -h localhost test_db # if you use postgres

python manage.py syncdb --settings test_db_settings.py
python manage.py migrate --settings test_db_settings.py (only if you use South)

Now we have DB that is used by TestRunner. We may run test:

REUSE_DB=1 python manage.py test

Updated

Are you sure that you use NoseTestSuiteRunner? Here is some code from django_nose.NoseTestSuiteRunner. As we can see if option REUSE_DB is set then teardown_database is disabled. If you want you can debug it for example set here a breakpoint to check you really use its Runner etc.

def teardown_databases(self, *args, **kwargs):
    """Leave those poor, reusable databases alone if REUSE_DB is true."""
    if not _reusing_db():
        return super(NoseTestSuiteRunner, self).teardown_databases(
                *args, **kwargs)
    # else skip tearing down the DB so we can reuse it next time

Dora

Thanx, I Am trying this now... and accept it as it works.... The problem was not creating the db the first time, but not deleting it after the test run. This given solution, with another settings file to run syncdb and migrate looks sufficient for my problem (so easy, I could have made it up myselve :) ) – Dominican

I have this in my settings.py: TEST_RUNNER = 'django_nose.NoseTestSuiteRunner' – Dominican

It is using nose for sure, it creates a nosetests.xml and using the nose arguments I put in the settings.py.... the problem was only if I run it with reuse_db="0" it tears down the database (as expected), if I use reuse_db="1" it does not create the database, so I couldn't run my tests. Creating it manually, using another settings file, solved this. – Dominican

Answer 2

I accepted the answer of Andrey Kaygorodov, because he lead me to the solution... reading his answer I thought (how easy, and how stupid of me...)

Anyway, putting this settings_test_db.py next to the settings with this content: (filling in the name of your project)

from <myproject>.settings import *

DATABASES = {
    'default': {
        'NAME': os.path.join(BUILDOUT_DIR, 'var', 'sqlite', 'unittest.db'),
        'ENGINE': 'django.db.backends.sqlite3', 
        'USER': '',
        'PASSWORD': '',
        'HOST': '',
        'PORT': '',
        }
}

And for sqlite you do not have to create a db. just run

python manage.py syncdb --settings <myproject>.settings_test_db  

(note: use the projectname, and without .py)

and to run the migrations

python manage.py migrate --settings <myproject>.settings_test_db

(only if you use South)

and set in your settings.py:

os.environ['REUSE_DB'] = "1" 

so you can use

python manage.py test

Dominican

Answer 3

by the way, in stead of using a separate test settings file... you can define a test anme in the same settings:

DATABASES = {
    'default': {
        'NAME': os.path.join(BUILDOUT_DIR, 'var', 'sqlite', 'test.db'),
        'TEST_NAME': os.path.join(BUILDOUT_DIR, 'var', 'sqlite', 'unittest.db'),
        'ENGINE': 'django.db.backends.sqlite3', 
        'USER': '',
        'PASSWORD': '',
        'HOST': '',  # empty string for localhost.
        'PORT': '',  # empty string for default.
        }
    }

Dominican