Python: spawn or thread for long running background process?

Question

Hi there,

I am planning to make a long running background process with Python but I am still unsure whether to use os.spawnle or thread. I've only read about it therefore I have not much experience with either spawn or thread. Is there any rule of thumb when to use which?

Thanks heaps

Answer 1

The obvious difference is that os.spawnle is used to start another process running a different program, whereas a thread would be executing code that's part of the same program. In fact, if your background process is some other program that already exists, then os.spawnle (or some other means of creating a separate process) is your only option; two threads in a program have to be running the same program.

If you wondering whether you should structure your own code to be run as separate processes or as separate threads, then take a look at some of the process v. thread questions like this one to decide which better fits what you're trying to do. In particular, consider what resources the processes/threads will need to share, what they will communicate with each other, and how robust each needs to be -- a thread that crashes will bring the rest of the process down with it, for example.

Answer 2

Be sure that you take the Global Interpreter Lock into account. If the long running process is CPU intensive, you should probably make it an independent process. If on the other hand, it's going to spend a lot of time blocking, then the GIL isn't really that big of a deal and you should be fine to make it a thread.

Also, if you don't need something in particular that os.spawnle provides, consider using the multiprocessing package from the standard library. It provides an interface similar to that of the threading package and is altogether easier to use than mucking around with manually spawning and tracking processes.