tags:

views:

47

answers:

2

I have an x64 (64-bit) COM dll. When trying to register it with Regsvr32 on Windows 7 - Regsvr32 crashes.

Regsvr32 is run under cmd with administrative priviliges ("run as administrator"), I tried both 32 and 64bit cmd.exe and regsvr.exe, even two different PCs and it is always the same.

Debugging the crashed Regsvr32 reports that buffer overrun has occured, what can be the problem?

All the same, but compiled as Win32 (32-bit) works fine and has no problems in registering

+1  A: 

Obviously, the problem is that you have a buffer overrun. Now the memory layouts of Win32 and Win64 processes will differ, especially with ASLR. You can't therefore say with certainty that the buffer overflow will have the same effects on Win32. This is especially true for something like Regsvr32, which will call one function in your DLL and exit. That limits the time in which the buffer overflow can damage data.

The solution is of course to just fix the buffer overrun.

MSalters
+1  A: 

Most likely the implementation of DllRegisterServer() in that DLL crashes when compiled for 64 bit.

If you have the source code for the DLL your best bet is to debug the implementation code as it executes and resolve the root cause of the problem. This can be any error typically occurring when code is not written in bitness-agnostic (32-bit/64-bit portable) manner.

sharptooth