It depends on what you want to do. I don't there are killer language features that 2.0 (Generics) and 3.0 (LINQ and Lambdas) had. C# 4 is more scenario based (see list below for some). If you want to:
- take advantage of the new features in WCF (default endpoints, improved WCF REST support, etc.);
- work with Entity Framework 4 and it's new features along with WCF Data Services to expose your EF model as a data service;
- take advantage of some of the multi-threaded enhancements (e.g. Tasks instead of using Threads);
- CodedUI tests (more of an IDE thing, not a language thing);
- etc.
...then maybe I would. (Most of my projects utilize WCF in some way, so I'm in the process of upgrading.)
I wouldn't upgrade just to upgrade. Upgrade if you have a business/technical reason to do so. You have to weigh the costs (less time coding while you upgrade your environment) with the benefits (latest language, and as one commenter said, happier developers).
Plus, the IDE is nice (although I've found it a little unstable at times). And as w69rdy's comment said, it's backwards compatible. You may want to give the IDE a try and still stay on .NET 3.5 just to try some of its features (like the extension manager and schema comparer).