How to debug Python code that is partially covered by passing unit tests but produces wrong results when being reused?

Question

I have a Python class and its methods are partially covered by unit tests like this:

class MyClass:

    def __init__(self):

       self.instancevar1 = None # some methods change this value
       self.instancevar2 = None # some methods change this value
       ...

    def method1(self, input_data):

        ...
        <self.instancevar1 and self.instancevar2 are used here>

        <return True or False>

...

class TestMyClass(unittest.TestCase):

    def test_method1(self):

        mc = MyClass()
        input_data = 'foo'
        self.assertEqual(mc.method1(input_data), True)

The tests are passing for simple cases like this.

I also have other classes that use instances of MyClass like this:

class MyOtherClass:

    def __init__(self):

        self.mc = MyClass()

Now, in some method of MyOtherClass, I have a situation like this:

class MyOtherClass:

    ...

    def some_method(self):

        ...
        result = self.mc.method1('foo') # result is expected to be True, but it is False

I think I understand the reason why this might happen. In the test case, I instantiate MyClass and call the test. In MyOtherClass, self.mc is referenced multiple times, misc methods of it are called and they probably change the internal state of the object (some values of instance variables), thus producing a different return value.

What are the suggested best practices for debugging such issues?

The fact that I have some tests for MyClass is good, but doesn't seem to help much here. Does this example contain some design flaws which cause my problems?

Answer 1

At first you need to find out if the MyClass object is in a correct state when some_method calls the defect method. If the state is correct, there is a bug in MyClass.method1, if it is not you need to search for the wrong state transition.

If there was a wrong state transition, you can add code to MyClass to detect the fault, and raise an exception or assert, when the wrong state is entered.

Answer 2

As Rudi said, there's either a bug in MyClass.method1() that causes it to return False when it shouldn't, or there's a bug in other parts of the code that puts mc in a state you didn't expect.

You can address both problems with additional unit tests:

• If there's a bug in MyClass.method1() when mc is in a particular state, you should add more unit tests to verify that mc.method1() returns the correct value for each state that mc is in.

• Also, add unit tests for MyOtherClass and other relevant parts of the code to ensure that self.mc is always in the state you're expecting, regardless of what operations you perform on the MyClass instance.

It's hard to come up with more specific advice than this since your example is fairly abstract, but I think you should get an idea about where to start looking and how to improve your tests to handle these issues better.