Depends entirely on who you are.
If your company is sufficiently established that it can get an internet merchant account from a bank, do so if your transaction volumes are greater than "very, very small". If you're turning over more than £1000 per month (or say $1000/month USD or something), then get a merchant account.
If you're NOT turning over that much, then first figure out how to get your turnover that high, because otherwise you're not going to be in business for long.
Internet merchant accounts not charge much less commission than the aforementioned "Hold on to the cash" PSPs, but they also give you the money sooner and are generally far more reasonable to work with.
There are many gateway providers which can work with the banks and the banks themselves typically offer their own. You can choose whatever combination best suits you from a technical perspective.
Disclaimer: This information is based on my limited experience integrating payment providers IN THE UK. It may not apply to other countries (or even here!)