I have an unmanaged DLL with a function that can run for a long time if the input parameter is a large value, sometimes that is desirable but not always.
How can I in c# call this function so that I can abort it when needed?
So far I have tried to put the call in a separate thread, but neither interrupt nor abort seem to stop the process, which runs at 100% CPU until the dll is done.
Is it possible to terminate the running dll code?
From the MSDN
If Abort is called on a managed thread while it is executing unmanaged code, a ThreadAbortException is not thrown until the thread returns to managed code.
So it seems, aborting the managed thread is not possible.
As an alternative you could try starting an unmanaged thread with Win32 APIs CreateThread(), WaitForSingleObject() and TerminateThread(). Use the signatures from here.
I didn't try this myself, but it surely introduces some risks, as you can not determine at which point of execution you are killing the thread. It might also lead to resource leaks. MSDN lists some of the side effects on the TerminateThread() reference.
In general I would advise against this kind of interruption. Maybe you have the possibility to change the unmanaged DLL to allow for a graceful abort.
Unmanaged code is only abortable if it is an "alertable wait state". It won't be when it is burning 100% cpu cycles. P/Invoking TerminateThread would work, assuming you could obtain the thread handle, which .NET makes very difficult. It won't help anyway, you'll leak the thread stack. At one megabyte, you'll quickly run out of virtual memory. Even if this only an occasional need, you're still liable to run into major problems since the thread has mutated global program state and you don't know how to restore it.
The only good way to abort unmanaged code is to run it in a separate process and shoot it in the head with Process.Kill(). The operating system will clean up the shrapnel. You'll need to write a little hosting program for the DLL and use one of the process interop facilities to talk to it. Sockets, named pipes, .NET remoting, WCF, take your pick.