If you're using p/invoke then you need to be sure that you've got 64bit versions of the DLLs available. This isn't an issue if you're calling MS supplied DLLs such as kernel32 as they've got the same name on the 32 and 64 bit platforms (bizarre, I know) and so your 64 bit app will implicitly link to the correct version. However, if you're using 3rd party DLLs and your 64bit app tries to link to a 32bit DLL you'll get a runtime exception.
This also means that if you're using in-process COM object you'll have a problem. A 64bit app can call an out-of-process COM object that's 32 bit (and vice-versa) as you're moving across a process boundary and COM will take care of the marshalling for you.
Other than this, the framework takes care of most things for you. You never really have to worry about pointer sizes and the clr types have explicit sizes (int is always 32 bits, long always 64 bits). In a pure .NET world there's very little to worry about. It's when you start moving outside the sandbox that you have to be more careful.