I'm working for a company who develops and selles an IDE. At the beginning we removed all project files away from the user directories. Most of the people were happy with this way. But from time to time someone asked if it is possible because they wanted to work also from there home system and use the version control as a fast file transfer utility synchronizing work and home place.
I can tell you that implementing this request was not easy!
The settings has to be clearly separated, for example if your IDE stores window coordinates in the settings file and you work on dual monitor system at work and a small notebook at home it is really bad. The IDE must also be able to provide a way to handle environment variable expansion in every file path. And finally of course it must be able to give setting files individual names or store them in different places - otherwise you will soon find out that 3 different developers on the project have 5 different opinions about fonts, colors, default values etc. because if this is not well done they will all use the same settings.
On the other hand many IDEs have huge amounts of data to store, some of them even store GUI configured unit test cases in project settings. In this case it is of course useful to reuse the files and check it into the version control system.
So you see it depends. Try it out and see how it works for your environment.