How Can I Catch a Mysterious Process Killer?

Question

We had this customer complaining out product was constantly crashing after running fine for 2-5 minutes. It took few days of guessing, but we came to the following conclusion:
When the process is terminating without leaving any traces (event logs/crash dumps) there are two options:
1. Our own process is calling TerminateProcess()
2. Someone else is killing us.

To catch a suicidal callstack (case 1) you just need to run "Adplus -crash -CTCFG …"
Alas, this didn’t help to resolve our customer situation.

We started to whisper to our customer ears, “Maybe someone is killing us, maybe a virus or something… ?!”.
The customer was skeptic (“ghosts / viruses / daemons?”), but just before getting ourselves kicked out, we were saved by some Sysadmin confession. He was reminded of a script they were periodically running monitoring our process. That script was wrongly thinking we are hanged and forced a process termination (Yack!).

Now comes the question:
For next time, we want a tool or a technique to catch who is killing our process? (I know there are some hacker tools to avoid getting killed, but I just want to log what’s the serial killer process PID/NAME.)

Are they any official/non-hacker ways to-do that?

Answer 1

You need to install a hook on system calls. you can also block these calls. Not entirely sure if its a non-hacker way :)

Answer 2

I would log every start of the application, if I were You. The rogue tool probably killed Your application in regular periods and it would raise Your eyebrow if You would see a start of Your app every 10 minutes or 10 seconds (or whatever it was set to).

As far as I know there is no silver bullet for this problem apart from something like tracking system calls and it's pretty hacky, but it should do the trick out-of-the-box.