x86-64 is a bit of a special case - for many architectures (eg. SPARC), compiling an application for 64 bit mode doesn't give it any benefit unless it can profitably use more than 4GB of memory. All it does is increase the size of the binary, which can actually make the code slower if it impacts on cache behaviour.
However, x86-64 gives you more than just a 64 bit address space and 64 bit integer registers - it also doubles the number of general purpose registers, which on a register-deficient architecture like x86 can result in a significant performance increase, with just a recompile.
It also lets the compiler assume that many extensions, like SSE and SSE2, are present, which can also significantly improve code optimisation.
However, if the app isn't performance sensitive, then none of this is really important either.