You certainly don't need your own delegate type - you can use EventHandler<TEventArgs> where TEventArgs is your specific EventArgs subclass.
Refactoring a large mess is always time consuming and annoying. If you change to using method group conversions it can make it easier in the future though:
// This...
foo.SomeEvent += new MyCustomEventHandler(SomeMethod);
// becomes this..
foo.SomeEvent += SomeMethod;
Then if the type of SomeEvent changes, you can change SomeMethod and the subscription will just work, without having to be changed again.
Whether you need several different EventArgs subtypes is a different matter - and impossible to say without knowing about your particular situation. If you need to pass a wide variety of pieces of information, it may indeed make sense.