Hi,
Here's a good article on Output caching: http://www.4guysfromrolla.com/articles/121306-1.aspx.
Generally, you seem to be looking at Page and Fragment caching. What you want to do is cache the Page, if you can, as that will give you the best performance benefit. But, if you have regions on the page that must change dynamically per user, eg: you are saying 'Hi {username}' at the top of the page, then you need to look at Fragment caching.
Fragment caching is not as effective as page caching, since the output has to be stitched together from cached info and dynamic info, but it's usually still MUCH better than NO caching!
It's a bit of an art, to tweak the caching depending on what the page does and the load on the database, but it can make a page load many Orders of Magnitude faster than non-cached.
FYI - if the db queries are killing the site you may want to also look at taming them and/or caching their output individually, so that you don't have to keep hitting the database for the same information.
Also understand the 'varyByParam' for caching is pretty useful too - say you have a page in 3 languages, you can cache a page for each language by using the varyByParam, as long as your Url some sort of language component that the varyByParam can pick up.
HTH,
Lance