I'm an avid Eclipse user (for Java), and I admit, it is very heavy.
You're not going to like all the features - though you can turn many of them off, at least in Eclipse.
I like automatic refactoring and I like auto completion (or content assist). But I also drive an automatic transmission. Sometimes I miss stick-shift, but most of the time I'm just trying to pound my way through something and it saves me time and effort. In the same way, sometimes I like picking my own methods, but I enjoy having Eclipse figure it out often.
To me the integrated debugger is the best feature of all these IDEs. I also like having tight SVN integration (or CVS/VSS/etc). I was not impressed by offerings outside the IDE.
I think in many cases, you have to find the one plugin that changes your life. To me it is things like the Java element search (better than grepping), and a plugin that I have written. For many people, something like Mylyn is perfect (it supposedly finds the most relevant files all the time). You can't really get something like it without an IDE, so you make the choice whether to use one or not.
In the end, the goal is to be productive and enjoy what you do. Deciding that you must or must not use a tool is more of a religion argument than something productive. Most people swear by Word, and I use LaTeX for everything. To each his own.