Difficult to debug embedded application

Question

I'm trying to debug an application on an embedded device running an old version of Linux/Qtopia. I asked for help on QT forums but the people there don't know about old software and embedded systems. I'd really like some help with debug strategies.

My program will crash after the main window has been constructed, i.e. some time into the event loop. But depending on the order of functions in the constructor, sometimes it will run only from the console and sometimes it will only run from the icon. Despite my best efforts I can't narrow down what is causing the problem.

There is no seg fault or signal but my program does not continue and the destructor does not get called. It seems to me that one of the first things that would happen in the event loop is a resize event and when this is called could vary if you ran from the console or icon. Also, the various widgets in my GUI would be initialised and drawn so that is also a potential source of error, if I haven't set up something properly.

My debugging options are limited as the area where the crash actually occurs is not under my control. I tried logging to a file and printing to stderr but this was no help. When I got to the state where it runs from the icon but not console, I tried running in gdb and strace but it ran OK - the classic problem of debug software initialising differently.

My next thought is to try to force a core dump and then analyse that. How do I force a core dump ? Is there a better strategy ?

Answer 1

Logging to a file or to a communication port (serial port, etc.) is probably the simplest way to see what is happening and maintaining the normal runtime (i.e. not in a debugger).

You say that logging to a file and printing to stderr was no help. Why not? Are you printing relevant debugging information to the file? Are you using the Linux/Qtopia sources and adding debug logging?

Assuming you have sources for all of the code you are running, it should be just a matter of adding debug logging in the right places to pinpoint where the problem is occurring.