No, you will not need to recompile for PostgreSQL to take advantage of the additional cores.
What will happen is that the Linux scheduler will now be able to select two or more (up to four) postgresql threads/processes to run all at the same time, basically they are working in parallel rather than having to wait on each other to get a slice of CPU time. This means you are able to process data faster since now four different queries can be processed at the same time rather than just the two you had previously.
PostGreSQL has no further tuning required to take advantage of multiple cores/physical CPU's and it is entirely up to the OS. You basically improved your performance for the cost of a new CPU.
If you are looking for information on tuning your PostgreSQL take a look at this post on tuning PostgreSQL on a dedicated server.
Since you now have more processes able to run at the same time, you may also want to consider upgrading the amount of RAM you have depending on what you currently have installed, the more the database is able to be stored in memory the faster all of the transactions and queries will be!