I'm never going to create a Visual Basic Project. Can I remove these from my IDE? Too many times I've created a new project not realizing I've been defaulted to the Visual Basic Projects and mistakenly create a Visual Basic Project instead of the C# project I meant to create.
Go to "Programs and Features" (Vista) or "Add or Remove Programs" (XP) and select Visual Studio. Click on "Uninstall or Change" (or the equivalent).
Then select "Add or Remove features" and under "Language Tools" remove "Visual Basic".
There may be other ways of messing with the templates directly, but that feels like the most appropriate way of saying you're not interested in VB.
EDIT: (Based on comments)
You can also change the default language, either by resetting all settings or by editing the vssettings file.
The templates are installed to C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\ProjectTemplates\VB.
You could try deleting that folder. There's also a cache folder in the IDE directory.
If that doesn't work or something screws up, you can always reinstall them by running a visual studio command prompt and typing "devenv /installvstemplates".
Go to Start -> Control Panel -> Add or Remove Programs -> Look for Visual Studio version you are working on and select 'Change/Remove' -> Remove Visual Basic from the features