If I have a solution with 10 projects. When I click "Clean Solution" does that just clean out the bin/debug in the main startup projects solution or the bin/debug in every projects directory?
It cleans every project contained in the solution, for the currently selected configuration (e.g. Debug, Release, etc).
To add to other answers, if you want a right click app, you can find the source code of clean source plus here.
This application does one thing. It adds an explorer shell menu to folders that when selected will recursively delete the contents of the bin, obj and setup folders. If you have a .NET project that you wish to share with someone, this is useful to remove the unnecessary stuff from the folder before you zip it up and send it off.
It cleans not everything but all projects that are configured to be built in your Solution properties. I.e. if you have solution that includes projects A and B and at some moment you check off Build in Solution properties, Configuration Properties section, for project B, next time when you make clean project B won't be cleaned.