There are of course several considerations, though at the core, the two languages are just different aspects of the same thing, so neither is objectively better.
If others will be helping you to maintain the application in the future, in very general terms it is easier to find people interested in coding in C#. Also, if you take any open source addons, you will mostly find them in C#. The language won't preclude you using them together, since they compile to the same thing, but you'd have to deal with having part of your application in one language, and another part in another.
If this will be a solo act, since you've just started programming I would not recommend you take on the burden of migrating from VB.NET to C#. It almost always turns out to be nontrivial, and besides you can learn a lot from maintaining an existing application in VB.