We're using Visual Studio 2010 (Premium edition if it matters), and pretty happy with it. However, We're never going to use the TFS features that's included in the IDE. (We're using Jira and Subversion, as it's not just Visual Studio that we work with, but also IntelliJ and a couple other IDE's.)
Is there any way to disable the TFS portions of the IDE? It's not a big deal or anything, just for the sake of "keeping things neat."
The main "TFS" parts of the IDE are in Team Explorer - Just don't install it.
Anything else you don't want/use, I'd advise you to simply ignore - VS has support for hundreds or even thousands of different things that you will probably never use, and you can't easily "clean" them all away.
In my experience the more you alter your installation of Visual Studio the more problems you will have with it. Every custom Option you set is another thing you have to repeatedly set every time you get a new PC or install a new VS. (Although it has improved a lot since import/export options became available and reliable). I used to spend about half a day setting up a visual studio to "work well", and now I just install it and use it. Ultimately I found that it was easier to just adjust my working practices (e.g. by relearning a few keyboard shortcuts etc) than to try to bend VS to my will.
In Visual Studio 2010, go to Tools->Options In the list, select Source Control. Set your Current source control plug-in: to None