On Linux (and many other Unix) systems strace. It allows me to debug system administration problems by looking at a command's interface to the operating system. It works on any binary (doesn't require compiling with debug symbols), and is especially useful if I don't (or can't be bothered to) have access to the command's source code. It can pinpoint configuration errors (where is this program looking for its input files?), uncover undocumented features (why is the program trying to open this file?), and performance problems (the command is writing to the file a character at a time). Similar tools are ltrace (Linux; works at the level of library calls), truss (FreeBSD and Solaris), and, of course, dtrace (Solaris, Mac OS X, and FreeBSD).