Django override_settings won't work for running multiple tests

Question

I am getting a bit odd behavior using the override_settings decorator. It basically works when I run the test alone, but won't work if I run the whole testing suite.

In this test I am changing the REST_FRAMEWORK options, because when running this suite I want to set the authentication settings, with the other tests do not use authentication:

@override_settings(REST_FRAMEWORK=AUTH_REST_FRAMEWORK)
class AuthTestCase(TestCase):
    @classmethod
    def setUpClass(cls):
        super(AuthTestCase, cls).setUpClass()
        cls.client = Client()

    def test_i_need_login(self):
        response = client.get('/')
        self.assertEqual(response.status_code, 401)

so if I do...

$ python manage.py test myapp/tests/test_auth.py

The settings are applied and works great!

but if run the whole testing suite like:

$ python manage.py test

The test will fail. It seems to me that these settings (or some objects) are being cached from other tests. I also have another class in another test file that uses a Client instance in similar way.

Environment: Python: 2.7 Django: 1.10

Edit:

The workaround I found to this problem was to use find to run the tests, it can be an alias or a script with...

find . -name 'test*.py' -exec python manage.py test {} \;

The downside is that the output of many tests gets piled up in the screen and it may create/destroy test database a few times. Unless you add options to the command like REUSE_DB if using django-nose.

Waxy

Answer 1

Well, there is a warning about this very situation.

Warning

The settings file contains some settings that are only consulted during initialization of Django internals. If you change them with override_settings, the setting is changed if you access it via the django.conf.settings module, however, Django’s internals access it differently. Effectively, using override_settings() or modify_settings() with these settings is probably not going to do what you expect it to do.

The first time you run the tests, you are running a specific test case so the override takes effect. The second time you run the test, you are running a whole suite and your specific testcase probably isn't the first one that's being run. So the above happens.

Pinole

There is way around this?? maybe by reloading the module. – Waxy

You will have to write a custom test runner but it's not going to be easy – Pinole

This warning is present in Django: 2.2 as well as the version in the original post. – Cohby