I've only used Eclipse once, for a few minutes, back when a 900MHz CPU was top-of-the-line. So I don't claim to know what it does or how well it does it.
That said, I see no problem at all with using a non-VS dev environment. Almost every C# developer at my company uses Emacs (there's also a VIM minority). We've got C# modes, we've got jump to definition, block folding, compilation modes, etc., all from our "editors". (I hesitate to use that word since I can't see a clear distinction between "editor" and "IDE".) There's not much special that VS can do that other environments can't, and 10+ years of experience and configuration easily trumps that.
The one thing special I can think of is the debugger: VS2005 had a standalone debugger, and VS2010 it's only available as part of the IDE (with a few new features), but that's it. But you can still launch VS2010 and connect to a running process just fine, even if you aren't otherwise using VS2010 for anything. The net effect of the slightly awkward debugger situation is mostly that I write more unit tests, and nobody seems to be complaining about that.
One thing that people like to bring up is "Intellisense". I've used VS in the past, and used Intellisense, and it's neat, but then I missed dynamic-abbrevs. Today I use Emacs and use dynamic-abbrevs and rarely miss Intellisense. I think dynamic-abbrevs are more useful once you know the language/stdlib you're working with. But I certainly don't think anybody has a monopoly on a One True Way to do code completion. Both are useful.
VS seems to do best the more you live only in Microsoft's world: .NET, TFS, etc. If you're using Microsoft-everything all the way down, it's probably pretty good, but the further you step away from that, the worse it gets. A more general environment like Emacs provides very good support for all kinds of things. (e.g., Do you want to learn a new UI when your next project uses a different VCS, or scripting language? The cost of that is only zero if your knowledge is worthless.)
So use whatever makes you productive, and don't let anyone tell you that you need Visual Studio Express to write a C# program.