The symbol packages are only any use for the version of Windows that you're currently running -- they install into C:\Windows\Symbols, and you can only have one installed at a time.
Use the Symbol Server instead.
Symbol Server
In Visual Studio, go to: Tools / Options / Debugging / Symbols. Add "http://msdl.microsoft.com/download/symbols" to the Symbol file (.pdb) locations list. You'll need to enter a cache directory in Cache symbols from symbol servers to this directory:. I use C:\WebSymbols. You'll plenty of space (I've got about 600Mb in mine), although you can clean the directory out periodically.
This will slow down your debugging sessions initially, until all the symbols you commonly need have been cached.
Alternatively, you can set this globally for all Windows debuggers by setting the _NT_SYMBOL_PATH environment variable to something like this: SRV*C:\WebSymbols*http://msdl.microsoft.com/download/symbols.
You can also (as part of your build process) collect your own .PDB files, using SYMSTORE.EXE (from Debugging Tools for Windows), and add that directory to this list.
Source Server
The .PDB files can have an (optional) reference to the source files that they were built from. As you mention, you can use this feature to download .NET sources, but it also works with C++ source files. We annotate our .PDB files with the SVN path and revision of the original source file. When we use "Go to Source", Visual Studio uses SVN to download the exact source code that the EXE was built with. This is really cool.
To annotate your PDB files, you'll need the tools from the srcsrv directory in the Debugging Tools for Windows.