The compiler takes each class through references. The bottom layer (no internal project references) has to be compiled first, then the layers that reference other internal projects, and so on. Therefore, yes, the compile time will be slightly increased due to the fact that the compiler has to sort the references and create multiple binaries.
On the other hand, most large projects SHOULD be separated into multiple projects and namespaces for ease of readability and navigation. It really depends on what you're doing, but you could potantially put 1000 classes in a single file, or multiple files, or multiple files in multiple projects. The amount of time (minimal) saved in the compile time won't compare to the time wasted looking for stuff in a poorly laid out solution.
With a solution of 2 projects with 5 classes, the compile time will be milliseconds different from a single project of 10 classes. The linking and referencing is the only real increase you'll see, and that is minimal.
EDIT: On another note, if you have a project large enough that you're noticing a real difference in compile time, you should probably be considering implementing some sort of continuous integration (see: http://en.wikipedia.org/wiki/Continuous%5Fintegration) environment, which will keep a current build ready for you, as well as letting you know if something's broken.