Building a COM object vtable in x86 assembly

Question

I am building a COM object in x86 assembly using NASM. I understand COM quite well and I understand x86 assembly pretty well, but getting the two to mesh is getting me hung up... (by the way, if you're thinking of attempting to dissuade me from using x86 assembly, please refrain, I have very particular reasons why I'm building this in x86 assembly!)

I am trying to build a vtable to use in my COM object, but I keep getting strange pointers, rather than actual pointers to my functions. (I'm thinking that I'm getting relative offsets or that NASM is embedding temporary values in there and they're not being replaced with the real values during linking)

The current interface I'm trying to build is the IClassFactory interface, with code as follows:

%define S_OK 0x00000000
%define E_NOINTERFACE 0x80004002

section .text

; All of these have very simple shells rather than implementations, but that is just until I can get the vtable worked out

ClassFactory_QueryInterface:
    mov eax, E_NOINTERFACE
    retn 12

ClassFactory_AddRef:
    mov eax, 1
    retn 4

ClassFactory_Release:
    mov eax, 1
    retn 4

ClassFactory_CreateInstance:
    mov eax, E_NOINTERFACE
    retn 16

ClassFactory_LockServer:
    mov eax, S_OK
    retn 8

global ClassFactory_vtable
ClassFactory_vtable dd ClassFactory_QueryInterface, ClassFactory_AddRef, ClassFactory_Release, ClassFactory_CreateInstance, ClassFactory_LockServer

global ClassFactory_object
ClassFactory_object dd ClassFactory_vtable

Note: This is not all of the code, I have DllGetClassObject, DllMain, etc. in a different file.

But when I assemble (using NASM: nasm -f win32 comobject.asm) and link (using MS Link: link /dll /subsystem:windows /out:comobject.dll comobject.obj), and examine the executable using OllyDbg, the vtable comes out with strange values. For example, in my last build, the actual addresses for the functions are as follows:

• QueryInterface - 0x00381012
• AddRef - 0x0038101A
• Release - 0x00381020
• CreateInstance - 0x00381026
• LockServer - 0x0038102E

But the vtable came out with these values:

• QueryInterface - 0x00F51012
• AddRef - 0x00F5101A
• Release - 0x00F51020
• CreateInstance - 0x00F51026
• LockServer - 0x00F5102E

These values look awfully suspicious... almost like the relocation didn't take. Also, the vtable comes out as 0x00F5104A, all of which are inaccessible memory addresses. (for informational purposes, these values come out different every time)

I tried doing the same thing in C++ using Visual Studio 2010 Express and everything comes out fine. So I'm assuming that it's just something that I'm missing in my assembly...

Can anyone point out to me why these values aren't coming out properly?

Answer 1

have you tried to build a stub COM interface in C and disassemble the result? That should give you a clue what is going wrong in your implementation.

Answer 2

Have you tried to also declare your globals as export? I haven't done x86 for a long time. But reading nasm's doc seems to imply you need to both global and export for the DLL relocation fixup to work.

Answer 3

I must apologize, the problem turned out to be my own fault... In all of the scuffle building the thing, I had removed the /dll from the linker invocation, causing it to be built as an EXE, not a DLL...

Let me explain this a little better for the next person who runs across this.

All Windows executables have a base address which is assumed to be the virtual address that the executable will be loaded into. Executables that are loaded into a running process in most cases will not be loaded at the "preferred" base address, because another DLL (or the application itself) is probably already occupying the address. For this reason, Windows PE executables use what is called a Relocation Table. The Relocation Table tells Windows which locations in the executable need to be rewritten in case of a relocation to a new base address.

However, with the advent of Virtual Memory, most linkers will omit the relocation table from EXEs as an optimization, because the executable will always be loaded at it's base address (unless it conflicts with the reserved kernel addresses, in which case it will fail to load all-together). So because I stopped compiling as a DLL, my executable was not being given a Relocation Table and as a result, would not load properly into running process' address space.