In Python, how do you programmatically execute unit tests stored in a string?

Question

The following code is used to execute doctests in a Google App Engine app. How would you do this for tests written as unit test asserts rather than as doctests?

#The solution and tests are untrusted code passed in to the GAE app. 
solution = 'b=5'
unittest = 'assertEqual(b, 5)'

#Here is the doctest version as a reference. 
solution = 'b=5'
doctest = '>>> b \n 5'

#Compile and exec the untrusted solution provided by the user. 
compiled = compile(solution, 'submitted code', 'exec')
sandbox = {}
exec compiled in sandbox

#Compile and exec each of the doctests
test_cases = doctest.DocTestParser().get_examples(doctest)
  for test in test_cases:
    if not test.want:
      exec test.source in sandbox

Answer 1

Methods of a class, such as unittest.TestCase.assertEqual, don't execute outside of the context provided by an instance of that class. So, a string like your 'assertEqual(b, 5)' is really a very, VERY bad case -- note that the string as written will never execute properly (you need to prepend, at the very least, somethink like 'self.', and then self needs to be made into an instance of the class, etc, etc).

I'm not sure exactly why you want to support such disastrous constructs, but, in case you're adamant that you do at all costs want to, then that's the general idea: make an instance of the unittest.Testcase class, prepend the instance-reference name and a dot to that string, and execute that compound string. Then, of course, you get into all sorts of other funny requirements, such as catching the exception that may get raised (since you don't really have a test runner to do all such housekeeping tasks for you). Yecch.