A possibly simpler MSVC++ 6 answer for a novice than the wikipedia article is as follows;
Symbols are links between an executable file being debugged and it's source, stored in a .PDB (symbolic information) file. If I am debugging an executable, or my EXE crashes and I end up in the debugger, and I have associated symbolic information, I will be able to view what's going on in terms of my C++ source code, assuming the source is available. If I don't have this information, I'll be shown x86 assembly / machine code.
For this reason, on test machines, it is often a good idea to supply the PDB and a debugger to the tester, as in the event of a crash, you'll be able to figure out why it happened. No PDB and debugger, and you'll have to recreate the crash on your development PC, which can be difficult.