ansaurus

Question

DLL monitoring.

Answer 1

A:

In general, this is a bad idea. Even if you have some set of captured parameters, without deep analysis of the DLL code you don't know what to do with those parameters and what ranges of parameters are accepted by certain methods. Example: if I call a method DoMathOperation(Add, 1, 2), you can mimic this call, but you won't be able to do DoMathOperation(Multiply, 2, 2) as you don't know that this is possible.

Eugene Mayevski 'EldoS Corp 2010-10-25 20:35:01

Answer 2

A:

The simplest approach has been to simply relocate the original dll, and create a new dll that you make yourself, with the same exports. This dll would LoadLibrary the old dll from the alternate location.

This doesn't quite apply here - the dll is exporting c++ class members which has two consequences: c++ classes have to be statically loaded as there is no c++ mechanism to 'glue' c++ function pointers (obtained via GetProcAddress) into a class instance.

This means your shim dll would be in the unfortunate place of having to both import, and export, and identical set of symbols.

The only way around this is to write your shim dll in two parts:

Shim1:

One part would get the name of the original dll, and would export the same class defintion the original dll exported:

 class __decldpec(dllexport) CCPCompressor {
  ...

Depends can crack the name decoration, or Undname.exe is distributed with Visual Studio.

This part would LoadLibrary() using an explicit path to shimdll2.dll located in some other folder, along with the original dll. GetProcAddress() would be needed to import functions exported by shimdll2.dll

Shim2:

The other shim dll would be located in a folder with the dll you are trying to intercept. This dll would have to import the class from the original compressor dll:

class __declspec(dllimport) CCPCompressor {
  ...

You can use the dll import library made by the first dll to actually link the symbols. Then its a case of exporting functions from shim2.dll that shim1.dll will call whenever a CCPCompressor method is called.

NB. Other things: your version of the CCPCompressor class will need to have, at least, a large dummy array as you can't know from the dll exports how big the application expects the class to be (unless you happen to have an actual header file describing the class).

To decompose the exported names to build a class definition: Open up the Visual Studio 20XX Command Prompt from the Start > Programs > Visual Studio 20XX -> Tools menu.

c:\...\VC>undname ?Clear@CCPCompressor@@QAEHXZ
Microsoft (R) C++ Name Undecorator

Undecoration of :- "?Clear@CCPCompressor@@QAEHXZ"
is :- "public: int __thiscall CCPCompressor:Clear(void)"

c:\...\VC>_

Do that for each function exported from the original dll (undname accepts some kind of text file to speed this process up) to find out how to declare a matching class def.

Chris Becke 2010-10-26 08:01:28

How would I find the same class definition the original dll exported?

None__ 2010-10-26 09:16:44

If you don't have it, then you need to invent it: use depends.exe to map out each function prototype, and add them all to your own class declaration.

Chris Becke 2010-10-26 10:05:50

I have put together the following but the ::New I am unsure about. class __decldpec(dllexport) CCPCompressor { private: CCPCompressor(void); ~CCPCompressor(void); public: int Clear(void); unsigned short Compress(unsigned char* const, unsigned short, unsigned char* const, unsigned short, int); void Delete(void); void Free(void); int Init(void); } static class CCPCompressor * __cdecl CCPCompressor::New(void); Do you have any really good URL's which are worth reading in relation to this?

None__ 2010-10-26 11:56:10

That class definition is looking good for the class shim dll. Unfortunately I can't think of any good sites that cover this, frankly i've never considered how to hook a c++ class from a dll before this post.

Chris Becke 2010-10-26 13:07:12

Answer 3

A:

Is using detours compatible with your requirements?

From the site:

Overview

Innovative systems research hinges on the ability to easily instrument and extend existing operating system and application functionality. With access to appropriate source code, it is often trivial to insert new instrumentation or extensions by rebuilding the OS or application. However, in today's world systems researchers seldom have access to all relevant source code.

Detours is a library for instrumenting arbitrary Win32 functions on x86, x64, and IA64 machines. Detours intercepts Win32 functions by re-writing the in-memory code for target functions. The Detours package also contains utilities to attach arbitrary DLLs and data segments (called payloads) to any Win32 binary.

Detours preserves the un-instrumented target function (callable through a trampoline) as a subroutine for use by the instrumentation. Our trampoline design enables a large class of innovative extensions to existing binary software.

We have used Detours to create an automatic distributed partitioning system, to instrument and analyze the DCOM protocol stack, and to create a thunking layer for a COM-based OS API. Detours is used widely within Microsoft and within the industry.

Daniel Mošmondor 2010-10-26 10:42:23

I could try but from the other post made it seemed like I couldn't use it for monitoring DLL's with classes. If you can tell me I can I will try it. Thanks.

None__ 2010-10-26 12:22:19

There is no application binary interface defined for c++ class method pointers. The result is method pointers cannot be marshaled via void* as they can vary wildly in size. Detours needs the address of a function with a matching calling convention to the function being hooked and I just can't see how to provide that.

Chris Becke 2010-10-26 13:34:42

Well, classes are just a collection of methods, from the your point of view - if you capture methods and log their calls with parameters, you will gain something.

Daniel Mošmondor 2010-10-26 13:35:07

@Chris: he didn't mention that he wants to monitor objects instance? Maybe if he wants only parameters, and ignore 'this', it could be feasible?

Daniel Mošmondor 2010-10-26 13:36:31

1. He wants to use the dll, so he needs to reverse engineer the class definition anyway. 2. When intercepting calls from a pre-existing exe to a dll, its not clear how to inject Detours using code into the process. 3. Even if that hurdle is crossed, calls made before the injection will be missed. 4. I CAN think of two potential ways to use Detours: 1. create a local shim class, and try and turn the class method pointers into plain memory addresses to pass to DetourAttach, 2. Detour to c-methods, with some custom asm to handle the __fastcall parameter packing.

Chris Becke 2010-10-26 14:02:03

ansaurus

tags:

views:

answers:

DLL monitoring.

related questions