Interruptible thread join in Python

Question

Is there any way to wait for termination of a thread, but still intercept signals?

Consider the following C program:

#include <signal.h>
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#include <pthread.h>
#include <stdlib.h>

void* server_thread(void* dummy) {
    sleep(10);
    printf("Served\n");
    return NULL;
}

void* kill_thread(void* dummy) {
    sleep(1); // Let the main thread join
    printf("Killing\n");
    kill(getpid(), SIGUSR1);
    return NULL;
}

void handler(int signum) {
    printf("Handling %d\n", signum);
    exit(42);
}

int main() {
    pthread_t servth;
    pthread_t killth;

    signal(SIGUSR1, handler);

    pthread_create(&servth, NULL, server_thread, NULL);
    pthread_create(&killth, NULL, kill_thread, NULL);

    pthread_join(servth, NULL);

    printf("Main thread finished\n");
    return 0;
}

It ends after one second and prints:

Killing
Handling 10

In contrast, here's my attempt to write it in Python:

#!/usr/bin/env python
import signal, time, threading, os, sys

def handler(signum, frame):
    print("Handling " + str(signum) + ", frame:" + str(frame))
    exit(42)
signal.signal(signal.SIGUSR1, handler)

def server_thread():
    time.sleep(10)
    print("Served")
servth = threading.Thread(target=server_thread)
servth.start()

def kill_thread():
    time.sleep(1) # Let the main thread join
    print("Killing")
    os.kill(os.getpid(), signal.SIGUSR1)
killth = threading.Thread(target=kill_thread)
killth.start()

servth.join()

print("Main thread finished")

It prints:

Killing
Served
Handling 10, frame:<frame object at 0x12649c0>

How do I make it behave like the C version?

Answer 1

Poll on isAlive before calling join. This polling can be interrupted, of course, and once the thread isn't isAlive, join is immediate.

An alternative would be polling on join with a timeout, checking with isAlive whether the timeout occurred. This can spend less CPU than the previous method.

Answer 2

Threads in Python are somewhat strange beasts given the global interpreter lock. You may not be able to achieve what you want without resorting to a join timeout and isAlive as eliben suggests.

There are two spots in the docs that give the reason for this (and possibly more).

The first:

From http://docs.python.org/library/signal.html#module-signal:

Some care must be taken if both signals and threads are used in the same program. The fundamental thing to remember in using signals and threads simultaneously is: always perform signal() operations in the main thread of execution. Any thread can perform an alarm(), getsignal(), pause(), setitimer() or getitimer(); only the main thread can set a new signal handler, and the main thread will be the only one to receive signals (this is enforced by the Python signal module, even if the underlying thread implementation supports sending signals to individual threads). This means that signals can’t be used as a means of inter-thread communication. Use locks instead.

The second, from http://docs.python.org/library/thread.html#module-thread:

Threads interact strangely with interrupts: the KeyboardInterrupt exception will be received by an arbitrary thread. (When the signal module is available, interrupts always go to the main thread.)

EDIT: There was a decent discussion of the mechanics of this on the python bug tracker here: http://bugs.python.org/issue1167930. Of course, it ends with Guido saying: " That's unlikely to go away, so you'll just have to live with this. As you've discovered, specifying a timeout solves the issue (sort of)." YMMV :-)

Answer 3

Jarret Hardie already mentioned it: According to Guido van Rossum, there's no better way as of now: As stated in the documentation, join(None) blocks (and that means no signals). The alternative - calling with a huge timeout (join(2**31) or so) and checking isAlive looks great. However, the way Python handles timers is disastrous, as seen when running the python test program with servth.join(100) instead of servth.join():

select(0, NULL, NULL, NULL, {0, 1000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 2000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 4000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 8000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 16000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 32000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
--- Skipped 15 equal lines ---
select(0, NULL, NULL, NULL, {0, 50000}Killing

I.e., Python wakes up every 50 ms, leading to a single application keeping the CPU from sleeping.