Debugging a program that doesn't start.

Question

How would I go about determining why a VB6 (or 5) application doesn't start? I can't attach a debugger to it because of that. I have IDA Pro Freeware at my disposal, but need some pointers as to where to start with it.

I don't have the source for this program, only the .exe. When I say it doesn't start, I double-click the .exe, and nothing happens - there is no process left running - nadda.

Running under admin it creates three empty folders, then exits silently before a UI is shown. OllyDbg tells me there was an 'Inexact floating point result', but I need to spend a few hours or days learning to interpret all the info OllyDbg gives me. My book on Advanced Windows Debugging should arrive on Monday or Tuesday as well.

Answer 1

In your Sub Main you should make sure there is an On Error Goto statement. The chances are an exception is being thrown and not caught. Try writing to a log file in the error handler and see what you get.

Answer 2

You might not have much luck without source, but you can look into this tip on MSDN for starting a debugger automatically. It's a little clumsy but it works with any program. Well behaved C++ applications stop at main, I believe. It can be a lifesaver for early errors.

Answer 3

I'd say grab a copy of OllyDbg and see when/why the application decides to terminate.

Answer 4

It seems like I answer with this a lot, but try looking at the Windows Events log. There is often information here that can be useful, such as a control, or dll your program depends on that was not found.

Answer 5

Are you saying the program starts but crashes before you can attach? If so try launching the program under the debugger.

In Visual Studio you can do this by going to

• File -> Open Project
• Select the EXE in question
• Hit F5

Answer 6

Have you tried running it within a cmd shell to see if it is returning any data to the console? Are any log files created?

Even if you can't attach a debugger since the program is not started, you can start it from within a debugger, OllyDbg will allow you to do so, as will any other debugger worth their salt. After which you can try debugging why the code is not starting in the first place.

Answer 7

Try checking event log for any related messages.

Use Filemon to check if the app tries to access any files ( no config? lets exit without error message.. )

http://technet.microsoft.com/en-us/sysinternals/bb896642.aspx

I believe there is similiar program for checking out what the app tries to do with registry.

Answer 8

First, check out Dependency Walker which will tell you if there are any DLLs missing that the app needs.

Second, what is the name of the executable? Try renaming it to MYDUMMY.EXE and run it again. There are some names that don't work because they are already in use. I once helped a new programmer who'd been trying to get his first app running for two weeks. It was called DISPLAY.EXE, which is already a windows module so it wouldn't load.

Answer 9

You can write a simple starter program that calls CreateProcess() with CREATE_SUSPENDED flag in the dwCreationFlags parameter.

It will start the program you debug and its thread will be immediately frozen. Then you can attach.

The master program may wait for say half a minute, then call ResumeThread(). You will have attached the debugger by that time.

Answer 10

Using WinDBG you can use "Open Executable" (or just do windbg {executable name} to connect to debugger instantly to the process during it's start up in ntdll and stop at a breakpoint before it's got your code and starting it running from there (type 'g') should give some information on the problem.

Answer 11

Another idea, try running in Compatibility Mode (right-clic, properties, compatibility Tab) with Windows 98 or Windows 2000.

Also give "Everyone" full access to the folder the program is creating this folders

Answer 12

I'd use the SysInternals Process Monitor, specifically following some of the advice from the Case of the Unexplained seminar video.

Start monitoring (everything) just before you launch it, then stop monitoring when you feel it's done being wrong, which should be in a few seconds. You will then have a detailed log of all the events that happened in all the processes. You can search for your process name, and find the non-successes, like maybe not having permissions, or getting some other complaint from the system which it isn't reporting to the user because you didn't code in a generic error trap with a dialog. Well, maybe even if you had it couldn't show the dialog for some reason. And the process monitor log will tell you about it.

It's like automatic logging without coding in a bunch of logging.

Answer 13

'Inexact floating point result' may sometimes (rarely, but happened to me) point at problem with (floating) number representation - like when program tries internally parse some hardcoded strings and assumes that decimals are always seprated by period or sometihing similar.

Try change system locales or just numbers or currencies format (decimal and thousands separators).

Answer 14

Well, like dlamblin mentioned, I'd grab ProcExp, FileMon, and RegMon from sysinternals, and set them up to monitor this process, then try and launch it. You will NO DOUBT get some information about one of the following:

• Config files it's trying to open/access
• Registry keys it's looking for
• Error levels returned from the process as it exits

Failing that, grab the free VBDecompiler, and have a look at what this thing is doing in it's startup code that might lead you somewhere.... If you are throwing an exception, and they have an

:Err

type label, that label might just fall through, and exit with no warning, or anything...

The VBDecompiler I've used was named "setup_free-VBDecompiler.zip". You can Google it to find it, there may even be a better version of it by now. But, don't expect it to decompile into the original source code, it can only make it's "best guess", but it could be handy to at least get you over this hurdle.... Be careful, and mindful/respectful of any EULA, that might prevent ANY reverse engineering...

Let me know if this info was of any help. If you still can't get anywhere, hook up with me, and send me the .exe and I can run it in my Norman Sandbox, which might give us more info on the function calls made, and what the app was trying to do...

Answer 15

Hi,

1.) If u have MS Visual Studio 2005 , it allows to run it on command line as follows:

devenv.exe is the executable(mostly in installation path e.g.(C:\Program Files\Microsoft Visual Studio 8\Common7\IDE)

run devenv.exe --help it gives the command line options I found some relevant options for your case:

devenv.exe /debugexe
Open the specified executable to be debugged. The remainder of the command line is passed to this executable as its arguments.

Hope that helps.

-AD