Normally, all directerories should have 755 (rwxr-xr-x) and all files 644 (rw-r--r--). If you want to set the most directories to not-writetable, you will need at least the directories listed in Help > System info > Directory Permissions. (Here you can also check which directories need to be changed, as non-writetable directories are shown in red.)
In some shared hosting environments, 755 / 644 isn't enough, as the owner of the file is not the (Web)Server User, but the FTP-User ... so either change it to 777 / 666 (not recommended, as you allow others to write your files), or get your FTP layer to work.
Another try: Joomla! does not work yet with the recent PHP 5.3. Can you use PHP 5.2? (Similar Problem in the Joomla Forum)