If you can create a good enough build system that it is trivial to recreate an exact build with just a checkout of code, I don't believe there is any need to store your builds in the repository.
For most of my stuff I do not store specific builds of my code, but I do store specific versions of the libraries my code relies on. I put a lot of effort in a few months ago to make it trivial to load in a tag and type "ant" and everything builds properly without relying on anything outside of the tree. (excluding the correct javac and ant)
Unfortunately, some of our codebase does not have as good a build system (ie, requires manual setting up of sdks and grabbing various external libraries and poking environment variables) and it would be difficult to recreate exactly a specific version of a build based on the repository (we are constantly moving forward and not really supporting old code, so the developers' workstation are set up close enough that we haven't been burnt yet by having to go back to an old branch before our current release) and in that case, we do store our release's builds (for the inevitable fat finger "oh no, I was on the wrong server doing some tests" or something equally insidious).