Patching out CALLL by replacing with NOPs works in user space but not in kernel space

views:

167

answers:

+1 Q:

Patching out CALLL by replacing with NOPs works in user space but not in kernel space

I have a device driver I want to patch. This device driver calls IOLog and I want to get rid of the logging.

If I replace the CALLL to IOLog with (the corresponding number of) NOPs inside the device driver (kext), the kernel crashes with what looks like a smashed stack ("Backtrace terminated-invalid frame pointer 0").

The same technique however, works fine in user-space (e.g. NOPping NSLogs inside an OS X binary).

What am I missing here?

+2 A:

You didn't explain whether you do cold patching (driver on disk) or hot patching (driver in memory). For in-memory patching, all kinds of issues might exist, such as the driver being executed when you patch it, the CPU(s) having cached portions of the code, etc. See the Intel manual section on self-modifying code.

For on-disk patching, it might be that you have a relocation record for the target address. So when the driver is loaded, the dynamic module loader will fixup the address of IOLog in the code, replacing it with the real address. This will overwrite your nop instructions.

Martin v. Löwis 2008-11-04 07:18:18

I am doing on-disk patching.If my NOPs were replaced as a side-effect of relocation, wouldn't then the driver function as before the patching?Relocation might play a part indeed, but I am yet to understand how it works. otool lists 390 relocation entries in the driver mach image.

diciu 2008-11-04 10:02:55

Relocation works by replacing the bytes in the instruction with the target address. So if the call is E8 xx xx xx xx, then it overwrites the xxs. If you replace E8 and all the xxs with nops, it leaves the first nop, and overwrites the others with the target, those bytes then become instructions.

Martin v. Löwis 2008-11-04 12:25:16

Ok, that makes sense. And it explains why when I used GDB to look at the device driver I saw call 0x0 while a symbol capable debugger showed call _IOLog. The symbol capable debugger must have looked at the relocation table.Thanks for the pointers, I'll keep digging.

diciu 2008-11-04 15:16:34

ansaurus

tags:

views:

answers:

Patching out CALLL by replacing with NOPs works in user space but not in kernel space

related questions