How to run django unit-tests on production database?

Question

Hi,

i'm starting the TDD development attitude and am writting unit-tests for my django application. I'm aware of fixtures and know that's the way tests should be executed, but for a given test i do need to execute it on the whole database, and json fixture for 10+ million row database is not something i'd like to handle, moreover, this test is "read-only".

So the question is how are you setting up your test suites to run on the production database? I imagine it could be as easy as adding the DATABASE_NAME setting in the setUp method of certain test. But the settings.DATABASE_NAME="prod_db" results in "NameError: global name 'settings' is not defined" while running the test. Moreover, there is a risk described in http://code.djangoproject.com/ticket/11987, that you can accidentally delete a production database.

So, how is it possible, or, even better, what is best practice, to run a single test of a test suite on a production database instead of temporary one?

Cheers in advance for any opinions!

Answer 1

First, if you're running it on the production database, it isn't much of a "unit" test.

It's a first-class batch job and needs to be treated like a first-class production batch job.

You cam't to use the Django test command for looking at production data. It always creates an empty database which is populated from fixtures in the TestCase.

You could make your production database processing a proper management command. This has the environment all properly configured so that your command can simply use the Django ORM to process your data.

The alternative is to be sure that you configure your settings. Either use the DJANGO_SETTINGS_MODULE environment variable or use the settings.configure() function to create an environment.

You can then import the models and do the processing you want to do against the production database.

You can call it "test" if you want to, but you're looking at production data, so it's got to be treated like production application with respect to getting the settings file and using the proper ORM configuration.

Answer 2

In case someone googles here searching for the solution for a given problem, here is the skeleton on how to perform unit tests on django production database. Check the django doc section here, for the file/directory structure, on where to put the given code. It should go in yourapp/management/commands/newcommandname.py, and both - the management and commands - folders should contain (empty) init.py files, what makes folders to be treated by python as modules. Test suite can by run as:

$python manage.py newcommandname

And here comes the code you should put in yourapp/management/commands/newcommandname.py :

from django.core.management.base import NoArgsCommand
import unittest

class Command(NoArgsCommand):
    help = """
    If you need Arguments, please check other modules in 
    django/core/management/commands.
    """

    def handle_noargs(self, **options):
        suite = unittest.TestLoader().loadTestsFromTestCase(TestChronology)
        unittest.TextTestRunner().run(suite)


class TestChronology(unittest.TestCase):
    def setUp(self):
        print "Write your pre-test prerequisites here"

    def test_equality(self):
        """
        Tests that 1 + 1 always equals 2.
        """
        from core.models import Yourmodel
        self.failUnlessEqual(1 + 1, 2)

Answer 3

A unittest is meant to test without any sideeffects. Though your test would nothing that is known as unittest. If you want to do it anyway, you can use a custom test runner that is setting up the database (or in your case using the existing db).

You can set the TEST_RUNNER setting in your settings.py file. The default is locate in django.test.simple.run_tests. You can look at the source here: http://code.djangoproject.com/browser/django/trunk/django/test/simple.py

Copy and paste the code in a new file and remove the following lines from the code:

connection.creation.create_test_db(verbosity, autoclobber=not interactive)
...
connection.creation.destroy_test_db(old_name, verbosity)

This will prevent django from creating a test database and reseting the database configuration of your settings file.