According to MS (I heard this from the MS project manager for VB.NET), the number of programmers for C# and VB.Net are roughly the same. So if you believe him, it is not really a career limiting move.
C# and VB.Net are moving into a co-evolving cycle. The main difference between the two will primarily be the language syntax, and not their capabilities. There will always be small differences, like I can't see VB.NET forcing the use of semi-colons, and I can't see C# having automatic statement terminators. Pretty much anything you can do in C# you can do in VB.NET now.
My advice to learn it is to sit down and try and learn the syntax. It's not like switching from C# to F# where there are fundamental differences in approach. If you find yourself thinking, "Man I could do this in C#," Chances are, with .Net 4.0, you can do it in vb.net....now if you only knew a qa programming board you could ask about how to do something in vb.net by giving a C# example..... :)