SetWindowsHookEx, CBTProc and g++3.4.5 (mingw)

Question

Hi,

I'm currently working on an application that needs to add a menu to each application's system menu. I can accomplish that for the existing windows with the EnumWindows function. For the new windows (applications starting up after mine) I'm trying to do this with windows hooks. More specifically with CBTProc. This is where I'm stuck. I've stripped about everything possible in the app, but I've got the impression the procedure in my dll just doesn't get called at all.

Here's the code for the dll:

#include <string>
using std::string;

#include <fstream>
using std::ofstream;

#include <windows.h>

// ---------------------------------------------------
extern "C"
{
  void log(const string & msg)
  {
    ofstream out("out.log", std::ios_base::app);
    out << msg;
    out.flush();
    out.close();
  }
  // ---------------------------------------------------  
  LRESULT CALLBACK CBTProc(int nCode, WPARAM wParam, LPARAM lParam)
  {
    log("CBTProc");
    return CallNextHookEx(0, nCode, wParam, lParam);
  }
  // ---------------------------------------------------  
}

I compile this with g++ 3.4.5 on a windows xp sp3 32bit machine:

g++ -shared -otest.dll test_dll.cpp

And here's the code for the application

#include <iostream>
using std::cout;
using std::cin;
using std::cerr;
using std::endl;

#include <string>
using std::string;

#include <windows.h>

typedef void (*func)();

void run()
{
  cout << "press enter to exit" << endl;
  cin.get();
}

void * loadProc(HMODULE mod, const char * procname)
{
  void * retval = (void *)GetProcAddress(mod, procname);
  if (retval == NULL)
    cerr << "GetProcAddress(" << procname << ") failed" << endl;
  return retval;
}

int main(int argc, char ** argv)
{
  HMODULE dll = LoadLibrary("test.dll");
  if (dll == NULL)
    {
      cerr << "LoadLibrary failed" << endl;
      return 1;
    }

  HOOKPROC proc = (HOOKPROC)loadProc(dll, "CBTProc@12");
  if (!proc)
    return 1;

  HHOOK callwnd = SetWindowsHookEx(WH_CBT, proc, dll, 0);
  if (callwnd == NULL)
    {
      cerr << "SetWindowsHookEx failed for CBTProc" << endl;
      return 1;
    }

  run();
  UnhookWindowsHookEx(callwnd);

  return 0;
}

I compile this with the same setup:

g++ -otest.exe test.cpp

When I run, I get no errors, but when I start up new applications I get nothing in my logfile.

Any ideas?

gr, ldx

Edit: spelling errors

Answer 1

I'd suggest you to check the following:

• ensure that your DLL has an export (can be checked with dumpbin tool). I don't know about g++, but in MSVC it is necessary to either use __declspec(dllexport) or explicitly state exports in DEF file.

• ensure that your host application uses the correct name of the exported function (the same as "dumpbin /EXPORTS test.dll" displays)

• keep in mind that you are are using relative file name out.log - when DLL gets loaded to other processes it would write relatively to the host process' current directory. For testing purposes it would be better to use OutputDebugString API and inspect results with DbgView tool.

Chances are that your solution works already.

PS: it's not generally a good idea to use STL in an injected DLL. Make sure you are aware of the risks.