If a Windows executable makes use of SYSENTER and is executed on a processor implementing AMD64 ISA, what happens? I am both new and newbie to this topic (OSes, hardware/software interaction) but from what I've read I have understood that SYSCALL is the AMD64 equivalent to Intel's SYSENTER. Hopefully this question makes sense.
+1
A:
If you try to use SYSENTER where it is not supported, you'll probably get an "invalid opcode" exception. Note that this situation is unusual - generally, Windows executables do not directly contain instructions to enter kernel mode.
Paul Baker
2009-10-29 13:35:10
Yeah, the usual thing on Windows is to call into a DLL which has the code to call the kernel.
Zan Lynx
2009-11-03 03:05:52
In fact, doing direct system calls will often get your exe detected as a virus.
Zan Lynx
2009-11-03 03:06:24
A:
As far as i know AM64 processors using different type of modes to handle such issues.
SYSENTER works fine but is not that fast.
A very useful site to get started about the different modes: Wikipedia
bastianneu
2009-10-29 13:43:57