Is TDD broken in Python?

Question

Hi!

Assume we have a class UserService with attribute current_user. Suppose it is used in AppService class.

We have AppService covered with tests. In test setup we stub out current_user with some mock value:

UserService.current_user = 'TestUser'

Assume we decide to rename current_user to active_user. We rename it in UserService but forget to make change to its usage in AppService.

We run tests and they pass! Test setup adds attribute current_user which is still (wrongly but successfully) used in AppService.

Now our tests are useless. They pass but application will fail in production.

We can't rely on our test suite ==> TDD is not possible.

Is TDD broken in Python?

Answer 1

Before the change, something should be different about the object's behavior, depending on the value of current_user. Let's call that something predicate(). And pardon my python; consider this pseudocode:

UserService.current_user = 'X'
assertFalse(obj.predicate())
UserService.current_user = 'Y'
assertTrue(obj.predicate())

OK? So, that's your test. Get it to pass. Now change the class under test so that current_user is renamed to active_user. Now the test will fail, either at the first assertion, or at the second. Because you're no longer changing the value of the field formerly known as current_user, so predicate will be false or true in both cases. Now you have a very focused test that will alert you when the class changes in such a way as to invalidate the other test's setup.

Answer 2

The problem really is not in TDD, nor Python. First of all, TDD does not give you a proof that when all your tests pass, your application is good. Imagine e.g. multiplyBy2() function, which can be tested with input 1,2,3 and output 2,4,8 and now imagine, you implemented multiplyBy2 as squaring. All your tests pass, you have 100% code coverage and your implementation is wrong. You have to understand, that TDD can only give you assurance, that once your test fails, something is wrong with your app, nothing more, nothing less. So as suggested in other answer, the problem is in that you do not have test that fails. If you had used some statically typed language, compiler would do that test for you and complain about using non-existent method. This does not mean you should use statically typed language, it just means you need to write more tests in dynamically typed language. If you are interested in enforcing correctness of the code, you should look at design by contract for ensuring correctness at least during runtime and formal specifications to have proofs for at least some algorithms, but this is I guess quite far from standard coding.

Answer 3

OK, I have found the solution. Python library Mock does what I want.

Below is the code I end up with.

Model and service definitions:

class User(object):
    def __init__(self):
        self.roles = []


class UserService(object):
    def get_current_user(self):
        return None # get from environment, database, etc.

    current_user = property(get_current_user)


class AppService(object):
    def __init__(self, userService):
        self.userService = userService

    def can_write(self):
        return 'admin' in self.userService.current_user.roles

Here's how to test can_write method of AppService with different users:

class AppServiceTests(unittest.TestCase):
    def test_can_write(self):
        user = User()

        @patch_object(UserService, 'current_user', user)
        def can_write():
            appService = AppService(UserService())
            return appService.can_write()

        user.roles = ['admin']
        self.assertTrue(can_write())

        user.roles = ['user']
        self.assertFalse(can_write())

If you rename property current_user only in class UserService, you'll get error when trying to patch the object. This is the behavior I was looking for.

This time TDD in Python is not broken. Python is better than I thought. Woo-hoo!