How do I map a segfault instruction pointer address from /var/log/messages to an address/function in my .map file?

Question

(My environment is 64-bit Ubuntu, my application is C++ compiled and linked with g++.)

When an application does something like divide by zero or run a asm("int $3") left in the code, one of the following gets logged via syslog to /var/log/kern.log and /var/log/messages:

Sep 10 18:06:47 VM kernel: [117194.123452] a.out[20288] trap divide error ip:45c59d sp:7fff65a91810 error:0 in a.out[400000+144000]
Sep 10 18:07:10 VM kernel: [117217.020833] a.out[20294] trap int3 ip:45c493 sp:7fff5cc559f0 error:0

In both those cases, the instruction pointer address points to something that I can easily look up in the .map file produced at link time (using the "-Wl,-Map,output.map").

But if I cause a seg fault, in this case by a call to memcpy() with the source set to NULL, the instruction pointer is so out of range, I have no idea how it is supposed to be mapped:

Sep 10 18:06:13 VM kernel: [117160.228587] a.out[20282]: segfault at 0 ip 00007f7e79209092 sp 00007fff831faf08 error 4 in libc-2.9.so[7f7e79185000+168000]

In this example, I would have expected the IP to be in the range of 0x445e70-0x445e7f, which is the location of memcpy() according to my .map file.

My question: What is the trick to interpreting the ip in this case?

Answer 1

According to the message it looks like it crashed inside memcpy(), from libc-2.9.so, which is mapped in to your process starting at 0x7f7e79185000. This is expected since memcpy is the function that is attempting to dereference the pointer. The instruction pointer looks valid since it is within the range of libc. If you were intending to override memcpy and call your own version, you may need to compile with -fno-builtin-memcpy.

Edit: You may be linking libc statically but according to the message you also have the libc shared library mapped into your process memory. You should see it listed in /proc/pid/maps while your program is running. It may be that you are linking with another shared library, such as libstdc++, and it depends on the libc shared library. As a result you have two versions of memcpy, and in this case it is calling the libc shared library version which is mapped at the high address. If you don't want the libc shared library then make sure you are linking all libraries statically; use the -static option at the beginning of your link line.