Ultimately, this comes down to what works best for you. Try emacs, IDE's, and other tools as you see fit and then use what you like and find easiest.
I have a co-worker who says he often has emacs and NetBeans up at the same time. He uses emacs for editing and NetBeans for investigating errors, building, testing, etc.
While I've dabbled off and on with Java for years, I'm just starting to get into it seriously due to a new job where it is the main development language. Previously, I've used emacs for most of my development; it's editing power is quite nice to have. However, for larger projects I find that for many things an IDE is nice also. So, I may adapt my co-worker's strategy of running both emacs and NetBeans simultaneously.
Prior to working on this job I'd worked a little with Eclipse, which I liked. However, NetBeans is the defacto standard where I'm now working, and has some good points, so I'm working on learning that.
Recently I learned of CEDET, which looks to be a way of wrapping an IDE around emacs. Unfortunately, I've not had time to investigate it yet, but you may find it worth a look.