The project template that you used to get the project started doesn't have very optimum settings. You'll get the clutter as a result. It is easily fixable. Start with Project + Properties, Compile tab. Make sure the Release build is selected, upper left combo box labeled Configuration.
The .pdb file contains debugging symbols. You don't need it for the Release build although you get slightly more informative exception messages. The stack trace will contain line numbers. You cannot trust them for a Release build though. Click Advanced Compile Options, Generate debug info = None.
The .xml file contains IntelliSense info, it will be generated when you use XML Documentation in your source code. Meant to be used for assemblies that are referenced in another project, quite pointless for an EXE project. Turn off the "Generate XML documentation file" option on the Compile tab.
The .vshost.exe file is a helper process for debugging your app. It hosts a custom version of CLR, configured differently to help with security issues while debugging. It also makes the output of Console.WriteLine() appear in the Visual Studio Output window. There's little point in having it created for the Release build. Select the Debug tab and uncheck the "Enable the Visual Studio hosting process" option.
After making these changes and rebuilding, you should only have the .exe file left in the bin\Release folder.
The slow startup is what's called a "cold start" of the .NET framework assemblies. It is caused by a slow or fragmented hard drive. Since the DLLs were never loaded before, the disk drive needs to dig through the GAC to find the files. You can probably improve it by defragging the disk. Cold starts are never as fast as warm starts though.
A classic trick, used by Microsoft Office and Adobe Acrobat, is to warm up the file system cache by loading their DLLs at login time. They are called "optimizer" in the Startup folder or Run registry key. Very annoying btw, they slow down other programs. You can do the same thing by writing your own little .NET program that doesn't do anything but create a few classes. Put a shortcut to it in the Startup folder.