Hi, I need to call a function (an LLVM JIT to be specific) from a C++ application. This call might fail or even signal abort() or exit(). How can I avoid or at least reduce effects on my host application? Someone suggested using fork(), however I need a solution for both windows and posix. Even if I would use fork() ... would it be possible for the two processes to communicate (pass some pointers around)?
You basically have to isolate the call that might fail spectacularly, so yes, you probably have to create a separate process for it. I'd actually be tempted to create a small executable just containing this particular call and the necessary supporting functionality and call that from your main executable. This gets you around the lack of fork()
on Windows and allows you to use the same mechanisms to communicate.
You can't pass pointers around between processes as they're not sharing the same address space. What I would do is have the spawned process reading data from stdin and write to stdout with the controlling process piping data into the child's stdin and reading from the child's stdout. Basically the way a Unix (command line) filter works. Another alternative if you're passing around a lot of data would be to write/read to/from a file on disk (better, a RAM disk) and communicate that way, but unless you're talking a lot of data, that's overkill.
As Eugen pointed out in the comments, you can also use shared memory if you want to pass pointers around or another inter-process communication mechanism depending on how much data you need to pass around. That said, choose the simplest possible method as nested executables like these aren't that easy to debug in the first place.