Off the top of my head (pre 4.0):
VB language "features" not supported in C#:
- Optional Parameters
- Late Binding
- Case insensativity
I'm sure there's more. Your question might get better answers if you ask for specific examples of where each language excels. VB is a currently better than C# when interacting with COM. This is because COM is much less of a headache when optional parameters are available, and when you don't have to bind to the (often unknown type) at compile time.
C# on the other hand, is preferable by many when writing complex logic because of its type safety (in that you can't bypass static typing) and its conciseness.
In the end, the languages are mostly equivalent, since they only differ on the fringes. Functionally, they are equally capable.
EDIT
To be clear, I'm not implying that VB doesn't allow static typing... simply that C# doesn't [yet] allow you to bypass static typing. This makes C# a more attractive candidate for certain types of architectures. In the 4.0 C# language spec, you can bypass static typing, but you do it by defining a block of dynamic code, not by declaring the entire file "not strict," which makes it more deliberate and targeted.