How to create good debugging problems for a contest?

Question

I am involved in a contest, and in one event we have debugging questions. I have to design some really good debugging problems in C and C++.

How can I create some good problems on debugging? What aspects should I consider while designing the problems?

Answer 1

Debugging is a broad scope, and it may be wise to reflect that in your questions. Without going into details, I can see the following categories :

Source-level debugging - no hints

Questions in this category just have source code, without any further hints on what's wrong. The actual bug can vary quite a lot here: from straightforward logic bugs like buffer overflows and counting errors to mistaken assumptions, via mathematical errors like rounding errors to just mistaken assumptions like assuming a particular endianness or padding.

Source-level debugging - problem stated

Questions in this category have source code, as well as desired versus actual output/behavior. E.g. "This program should print 42, but instead prints Out of Memory. Why?"

Crashed code

Questions in this category come not just with source code, but also with a crash dump.

Answer 2

My brainstorming session:

Memory leaks of the subtle sort are always nice to have. Mess around with classes, constructors, copy-constructors and destructors, and you should be able to create a difficult-to-spot problem with ease.

One-off errors for array loops are also a classic.

Then you can simply mess with the minds of the readers by playing with names of things. Create variables with subtly different names, variables with randomized (AND subtly different) names, etc. and then let them try and spot the one place where you've mixed up length and lenght. Don't forget about casing differences.

Calling conventions can be abused to create subtle bugs too (like reversing the order of parameters).

Also let's not forget about endless hours of fun from tricky preprocessor defines and templates (did you know that C++ templates are supposedly Turing-complete?) Metaprogramming bugs should be entertaining.

Next idea that comes to mind is to provide a correct program, but flawed input data (subtly, of course). The program will then fail for the lack of error checking, but it will be some time until people realize that they are looking for problems in the wrong place.

Race conditions are often a difficult to reproduce and fix, try to play with multithreading.

Underflows/overflows can be easily missed by casual inspection.

And last, but not least - if you're a a programmer, try remembering what was the last big problem that you spent two weeks on solving. If you're not a computer programmer, try to find one and ask them. I'm a .NET programmer, so unfortunately my experiences will relate little to your requirement of C/C++.

Answer 3

Real world debugging would include find synchronization problems and problems between managed/unmanaged boundary, so please consider c/c++/c# as an option.

Or for real fun, consider using just c# and finding memory leaks.

Also, you will need to mention which tools are allowed to be used. On windows, there are literally dozens of debugging tools available.

Answer 4

For some simple "find the bug in this source code" excercises, check out PC-lint's bug of the month archive.

Answer 5

I'll add to the answers above that another form of bugs is the incorrect use of some library or API code. Superficially everything looks ok, but there is some caveat (e.g., a precondition or a limination) that one is not aware of. Interactive debuggers are not as effective by themselves in these situations because they don't expose that information to you (it's often hidden in the documentation).

For example, I did in the past a study of this stuff. I gave people code that used (a messaging API in Java), where the error was that the program was getting stuck as soon as you tried to receive a message. Debugging this interactively was almost impossible. They had to manually figure out what was going on, and realize that one of the queues wasn't set up correctly.

These sort of bugs are actually quite common.

Answer 6

In addition to what's above, consider side effects. For example:

// this function adds two ints and returns the sum

int add_em(int &one, int &two)
{
   two += one;
   return two;
}

As you can see, this code modifies the two variable, although the comment doesn't mention that...

Answer 7

Article. Problems of testing 64-bit applications