debugging loop c++ undefined variable, what type? Hoglund

Question

Hi,

I have been reading one of the Hoglund titles and I though, reading great, but can I make it work? Why do they provide non-working examples in books?

#include "stdafx.h"
#include <cstdio>
#include <windows.h>
#include <winbase.h>
#include <tlhelp32.h>

int _tmain(int argc, _TCHAR* argv[])
{
    HANDLE hProcess;
    DEBUG_EVENT dbg_evt;
    int aPID;

    if(argc != 2)
    {
        printf("wrong number of params\nusage %s<pid>\n", argv[0]);
        return 0;
    }

    //load the ptr to fDebugSetProcessKillOnExit
    fDebugSetProcessKillOnExit = (DEBUGSETPROCESSKILLONEXIT)
    GetProcAddress(GetModuleHandle("kernel32.dll"), 
    "DebugSetProcessKillOnExit");
    if(!fDebugSetProcessKillOnExit)
    {
        printf("[!] failed to get fDebugSetProcessKillOnExit function!\n");
    }

    aPID = atoi(argv[1]);
}

I am getting two error messages:

1. fDebugSetProcessKillOnExit is an undeclared identifier
   What type should it be?
2. "Error 4 error C2664: 'GetModuleHandleW' : cannot convert parameter 1 from 'const char [13]' to 'LPCWSTR'
   What type should the fDebug... be? Why doesn't the aPid = atoi... line work?

Should the project be compiled in C instead of C++, as this is exactly as it is in the book?

Thanks, R.

Answer 1

Taking this from MSDN:

BOOL WINAPI DebugSetProcessKillOnExit(__in  BOOL KillOnExit);

you can declare the function pointer as:

BOOL (*fDebugSetProcessKillOnExit)(BOOL) = /* ... */;

or ease your eyes by using typedef:

typedef BOOL (*DebugKillPtr)(BOOL);
DebugKillPtr fDebugSetProcessKillOnExit = /* ... */;

Function pointers can be somewhat confusing, InformITs guide on them should help with that.

Additionally, you are using a Unicode build. You can either use GetModuleHandle(L"Kernel32.dll") or _T() etc. or set your project to use the multi-byte-character-set (project properties -> configuration -> general -> character set).

The unicode-character-set is also the reason why the atoi() statement can't work:
argv is an array of _TCHAR*s, and _TCHAR is wchar_t for unicode-builds. atoi() however expects a const char* argument and you are handing it a wchar_t*.
So you can either again use a multi-byte-character set, convert the string or use Microsofts _wtoi()/_ttoi().

To ease switching between character sets and stick with the style of the book, prefer the _T* and _t* versions.

Answer 2

1. The declaration for fDebugSetProcessKillOnExit is missing; probably it should be something like

 

DEBUGSETPROCESSKILLONEXIT fDebugSetProcessKillOnExit;

where DEBUGSETPROCESSKILLONEXIT should be a typedef to the prototype of that function.

1. You are compiling in Unicode and the book "thinks" that you're compiling in ANSI; you should change the strings passed to the APIs to generic strings with the _T() macro.

So, to sum everything up, you should simply change one line and add the typedef:

typedef BOOL (*DEBUGSETPROCESSKILLONEXIT)(BOOL);
//load the ptr to fDebugSetProcessKillOnExit
DEBUGSETPROCESSKILLONEXIT fDebugSetProcessKillOnExit = (DEBUGSETPROCESSKILLONEXIT) GetProcAddress(GetModuleHandle(_T("kernel32.dll")), _T("DebugSetProcessKillOnExit"));