We actually have explicit delays set in our continuous integration server (Hudson) so that the builds don't "break" if multiple commits are needed in order to bring the repository back to a usable state (I'm talking about multiple commits over the course of like a minute). For example, you may have multiple projects that depend on each other, and they each need to be committed, or you may have different pieces of code that should be committed separately, as the changes are logically independent.
One of the primary goals of continuous integration is to make sure that nothing breaks inadvertently, and in particular make sure that if something does break, someone finds out as soon as possible. It is a whole lot quicker and easier to fix an issue at a point in time closer to when the issue was created, than later on when a developer finally gets around to doing a build.
Additionally, the mere presence of a continuous integration (build) system has the tendency to keep developers on their best behavior.
So as long as your builds aren't taking hours, that's probably fine.