How to find which type of system call is used by a program

Question

I am working on x86_64 machine. My linux kernel is also 64 bit kernel. As there are different ways to implement a system call (int 80, syscall, sysenter), i wanted to know what type of system call my machine is using. I am newbie to linux. I have written a demo program.

#include <stdio.h>
int main()
{
  getpid();
  return 0;
}

getpid() does one system call. Can anybody give me a method to find which type of system call will be used by my machine for this program.. Thank you....

Answer 1

Under linux, you can use strace to record which system calls are made by a particular process.

Answer 2

victory:~ # gcc getpid.c -o getpid -g
victory:~ # gdb getpid
<snip>
(gdb) break main
Breakpoint 1 at 0x400540: file getpid.c, line 4.
(gdb) run
Starting program: /root/getpid 

Breakpoint 1, main () at getpid.c:4
4     getpid();
(gdb) disassemble
Dump of assembler code for function main:
0x000000000040053c <main+0>:    push   %rbp
0x000000000040053d <main+1>:    mov    %rsp,%rbp
0x0000000000400540 <main+4>:    mov    $0x0,%eax
0x0000000000400545 <main+9>:    callq  0x400440 <getpid@plt>
0x000000000040054a <main+14>:   mov    $0x0,%eax
0x000000000040054f <main+19>:   leaveq 
0x0000000000400550 <main+20>:   retq   
End of assembler dump.

Looks like our call to getpid() is actually a library call. Let's set a breakpoint there and continue.

(gdb) break getpid
Breakpoint 2 at 0x7ffff7b29c00
(gdb) cont
Continuing.

Breakpoint 2, 0x00007ffff7b29c00 in getpid () from /lib64/libc.so.6
(gdb) disassemble
Dump of assembler code for function getpid:
0x00007ffff7b29c00 <getpid+0>:  mov    %fs:0x94,%edx
0x00007ffff7b29c08 <getpid+8>:  cmp    $0x0,%edx
0x00007ffff7b29c0b <getpid+11>: mov    %edx,%eax
0x00007ffff7b29c0d <getpid+13>: jle    0x7ffff7b29c11 <getpid+17>
0x00007ffff7b29c0f <getpid+15>: repz retq 
0x00007ffff7b29c11 <getpid+17>: jne    0x7ffff7b29c1f <getpid+31>
0x00007ffff7b29c13 <getpid+19>: mov    %fs:0x90,%eax
0x00007ffff7b29c1b <getpid+27>: test   %eax,%eax
0x00007ffff7b29c1d <getpid+29>: jne    0x7ffff7b29c0f <getpid+15>
0x00007ffff7b29c1f <getpid+31>: mov    $0x27,%eax
0x00007ffff7b29c24 <getpid+36>: syscall 
0x00007ffff7b29c26 <getpid+38>: test   %edx,%edx
0x00007ffff7b29c28 <getpid+40>: mov    %rax,%rsi
0x00007ffff7b29c2b <getpid+43>: jne    0x7ffff7b29c0f <getpid+15>
0x00007ffff7b29c2d <getpid+45>: mov    %esi,%fs:0x90
0x00007ffff7b29c35 <getpid+53>: mov    %esi,%eax
0x00007ffff7b29c37 <getpid+55>: retq   
End of assembler dump.

Buried in the getpid() library is the syscall assembler instruction. This is an AMD64 instruction that supports a fast context switch to ring0 for the purpose of system calls.

Answer 3

One way is to use the gdb to step through the machine code (using stepi) until you get to the instruction that initiates the system call. Because different machines put the instruction in different places (sometimes in the system call wrapper itself and sometimes in a function called by the system call wrapper), I can't predict where exactly the instruction will be.

For example, on one old machine, getpid itself did a int 0x80 while in a newer machine, getpid does a call *gs:0x10 which brings it to __kernel_vsyscall which does a sysenter