MSDN seems to indicate that the Cxxx casts for specific types can improve performance in VB .NET because they are converted to inline code. For some reason, it also suggests DirectCast as opposed to CType in certain cases (the documentations states it's when there's an inheritance relationship; I believe this means the sanity of the cast is checked at compile time and optimizations can be applied whereas CType always uses the VB runtime.)
When I'm writing VB .NET code, what I use depends on what I'm doing. If it's prototype code I'm going to throw away, I use whatever I happen to type. If it's code I'm serious about, I try to use a Cxxx cast. If one doesn't exist, I use DirectCast if I have a reasonable belief that there's an inheritance relationship. If it's a situation where I have no idea if the cast should succeed (user input -> integers, for example), then I use TryCast so as to do something more friendly than toss an exception at the user.
One thing I can't shake is I tend to use ToString instead of CStr but supposedly Cstr is faster.