What are the possible pitfalls in porting Psyco to 64-bit?

Question

The Psyco docs say:

Just for reference, Psyco does not work on any 64-bit systems at all. This fact is worth being noted again, now that the latest Mac OS/X 10.6 "Snow Leopart" comes with a default Python that is 64-bit on 64-bit machines. The only way to use Psyco on OS/X 10.6 is by recompiling a custom Python in 32-bit mode.

In general, porting programs from 32 to 64 bits is only really an issue when the code assumes a certain size for a pointer type and other similarly small(ish) issues. Considering that Psyco isn't a whole lot of code (~32K lines of C + ~8K lines of Python), how hard could it be? Has anyone tried this and hit a wall? I haven't really had a chance to take a good look at the Psyco sources yet, so I'd really appreciate knowing if I'm wasting my time looking into this...

Answer 1

Since psyco is a compiler, it would need to be aware of the underlying assembly language to generate useful code. That would mean it would need to know about the 8 new registers, new opcodes for 64 bit code, etc.

Furthermore, to interop with the existing code, it would need to use the same calling conventions as 64 bit code. The AMD-64 calling convention is similar to the old fast-call conventions in that some parameters are passed in registers (in the 64 bit case rcx,rdx,r8,r9 for pointers and Xmm0-Xmm3 for floating point) and the rest are pushed onto spill space on the stack. Unlike x86, this extra space is usually allocated once for all of the possible calls. The IA64 conventions and assembly language are different yet.

So in short, I think this is probably not as simple as it sounds.

Answer 2

+1 for "... how hard could it be?".

Take a look here: http://codespeak.net/svn/psyco/dist/c/i386/iprocessor.c

All that ASM would have to be ported, and there are assumptions all over the place about the underlying processor.

I think it's fair to say that if it were trivial to port (or not too difficult), it would already have been done.

Answer 3

Christian Tismer, one of the Psyco developers also seems to disagree with the "how hard could it be" - assumption (quoted from here):

Needs to come to x86-64? Why that! Seriously, I would love to do that, but this would be much harder than anybody would expect. Due to the way psyco is written, it would be really half a rewrite to releave it from its 32-bitterness. The 32 bit assumption is implicitly everywhere. It would be straight forward, if the memory model was all 64 bit. But no intel platform is that simple. so long, and thanks for all the fish -- chris

and

hum. It would cost at least 3 or 4 months of full-time work do do that, if not more. I doubt that I can get sponsorship for that.

If you want more details (firm inside knowledge of Psyco probably needed) I guess you can always try to ask on one of the psyco mailing lists ...

Answer 4

Psyco assumes that sizeof(int) == sizeof(void*) a bit all over the place. That's much harder than just writing down 64bit calling conventions and assembler. On the sidenote, pypy has 64bit jit support these days.

Cheers, fijal