Why do you assume they aren't important to the running of the application? What if they are? What are the 'other things' you mention as well?
If you removed the default error messages, a user might legitimately encounter that error, then there would be no message displayed to them, leaving them even worse off. (As another answer mentioned these are most likely from the C runtime library, and most likely your program does need that!)
The XML could be an embedded manifest that specifies some of the DLL requirements. This helps prevent the 'DLL hell' situation where programs depend on different versions of a DLL with the same name. Amongst other things, it specifies the precise version information of DLLs, so you never run in to that ugly situation. Again, this seems to be something you would want.
The libraries and compiler settings for most C++ compilers will most likely add "junk" (I'd just call it library data) because of all the useful features they provide, saving you from having to solve those problems all over again yourself. If you still don't like any of that - stop using compilers, and start using assembler. Then you can specify exactly what is and isn't in the final compiled program, but you'll probably end up reinventing the wheel...