.htaccess help: changing /register.php into /register

Question

I have no idea about anything to do with .htaccess, apart from the basics of what it's used for. In this case I need it to rewrite URLs.

I have a set of pages:

index.php
register.php
login.php
donate.php

And I wanted to change them into URL's like this:

http://localhost/
http://localhost/register
http://localhost/login
http://localhost/donate

As far as I know using .htaccess is the best way to do this, instead of putting them all in individual sub folders?

If .htaccess is the best way, could you provide the contents of .htaccess that would do this? Or at least give me some direction.

Thank you. :)