check for threads still running after program exits

Question

Hello

gcc 4.4.3 c89 pthreads

I use valgrind for checking memory errors.

I am just wondering if there is any tool for linux that can detect running threads that haven't been terminated after the program finishes.

I am running a multi-thread application and need a tool to make sure all threads have finished.

Many thanks for any suggestions,

Answer 1

Correct me if wrong, but a program is not finished until all running threads have ended.

Answer 2

If they're threads (rather than processes) then you just need to check for your process stll running because threads run inside a process.

You can check if a process is running with ps -ef then pipe the result into grep to find your specific process.

Answer 3

If the program has terminated (because the initial thread returned from main(), some thread called exit(), or a fatal signal was recieved by the process) then you are guaranteed that all threads have been terminated with extreme prejudice.

---

If you want to write your program so that it ensures that all its threads have exited before main() exits, then you need to loop over all your threads at the end of main(), calling pthread_join() on each one. (This also means that you shouldn't create your threads detached, or detach them).

Answer 4

If you plan to use Boost.Threads library, then you can use the .join() method.

For example:

#include <boost/thread/thread.hpp>
#include <iostream>
void hello()
{
  std::cout <<
    "Hello world, I'm a thread!"
    << std::endl;
}

int main(int argc, char* argv[])
{
  boost::thread thrd(&hello);
  thrd.join();
  return 0;
}

Answer 5

If you can not use C++ and therefore KMan's answer, then you can also join detached pthreads using the "C" API. (Joining means to wait for the detached threads to finish their work.)

See the pthread tutorial.

Answer 6

The existance of the process, that is if there is any thread still running, can be checked with waitpid.

If you just want your process to continue with all the threads, but you don't need the one of main anymore you can end that thread by pthread_exit. Other than an explicit exit or a simple return this will not terminate your other threads.

Answer 7

If you want an external means to observe the threads in execution for your process, on Linux you can look in /proc/(pid)/task. That's the method tools like ps(1) or top(1) use.

See http://linux.die.net/man/5/proc

Answer 8

There is a simple trick in this similar question: http://stackoverflow.com/questions/1695106/multiple-threads-in-c-program

If you call pthread_exit from main, your process will not exit until all the other threads complete.

Answer 9

You're missing out on the important part:

A program cannot exit unless all its threads are terminated.

What you should do, however, is pthread_join() on all the threads before exiting. This ensures that all threads terminated, and will allow you to free() all their respective pthread_ts, so that you do not leak memory from them.

Have that said, valgrind can give you a comprehensive view on threads you haven't cleaned up after. Run it with --leakcheck=full and make sure you are not leaving various structs behind you. Those will indicate there is a thread you haven't completely terminated properly.

Answer 10

Original answer was updated to address pthread_exit() scenario.

Assuming you want to tell whether all threads were pthread_join()-ed properly before you return from main(), there are a few ways:

1. Run it under the gdb and break on the last line of main(), then look at the output of "threads" command. There should only be main thread.

2. Make a shared library that overrides pthread_create with a wrapper that keeps a counter of how many threads are started. Thread wrapper increments a counter and calls the actual thread function, and a function registered with pthread_create_key() will decrement it when a thread returns or exits. Library destructor will check if the counter is zero, which means that all of them were terminated. Use it with your executable with LD_PRELOAD=checker.so ./your_executable (no code modification necessary).

   Tested on Debian 5.0.5.

   checker.c

   #define _GNU_SOURCE
   #include <pthread.h>
   #include <stdio.h>
   #include <dlfcn.h>
   #include <stdlib.h>
   
   
   /* thread-local storage key */
   static pthread_key_t tls_key = 0;
   static int counter = 0;
   static pthread_mutex_t g_mutex;
   
   
   /* TLS destructor prototype */
   void on_thread_end(void*);
   
   
   void __attribute__ ((constructor))
   init_checker()
   {
       pthread_mutexattr_t attr;
       pthread_mutexattr_init(&attr);
       pthread_mutex_init(&g_mutex, &attr);
       pthread_mutexattr_destroy(&attr);
       pthread_key_create(&tls_key, &on_thread_end);
   }
   
   
   void __attribute__ ((destructor))
   finalize_checker()
   {
       int remain;
       pthread_mutex_lock(&g_mutex);
       remain = counter;
       pthread_mutex_unlock(&g_mutex);
       pthread_mutex_destroy(&g_mutex);
       if (remain)
           fprintf(stderr, "Warning: %d threads not terminated\n", remain);
       pthread_key_delete(tls_key);
   }
   
   
   /* thread function signature */
   typedef void* (*ThreadFn)(void*);
   
   
   struct wrapper_arg
   {
       ThreadFn fn;
       void* arg;
   };
   
   
   /* TLS destructor: called for every thread we created
      when it exits */
   void
   on_thread_end(void *arg)
   {
       free(arg);
       pthread_mutex_lock(&g_mutex);
       --counter;
       pthread_mutex_unlock(&g_mutex);
   }
   
   
   static void*
   thread_wrapper(void *arg)
   {
       void *ret;
       struct wrapper_arg *warg;
   
   
   warg = (struct wrapper_arg*)arg;
   
   
   /* Thread started, increment count. */
   pthread_mutex_lock(&amp;g_mutex);
   ++counter;
   pthread_mutex_unlock(&amp;g_mutex);
   
   
   /* set thread-specific data to avoid leaks
    * when thread exits
    */
   pthread_setspecific(tls_key, arg);
   
   
   /* Run the actual function. */
   ret = (*warg-&gt;fn)(warg-&gt;arg);
   
   
   /* Thread finishes, TLS destructor will be called. */
   
   
   return ret;
   
   
   }
   
   
   /* pthread_create signature */
   typedef int (*CreateFn)(pthread_t*,const pthread_attr_t*,ThreadFn,void*);
   
   
   /* Overriding phtread_create */
   int
   pthread_create(
       pthread_t *thread,
       const pthread_attr_t *attr,
       ThreadFn start_routine,
       void *arg)
   {
       CreateFn libc_pthread_create;
       struct wrapper_arg *warg;
   
   
   /* Get a handle to the real function. */
   libc_pthread_create
       = (CreateFn)dlsym(RTLD_NEXT, "pthread_create");
   if (!libc_pthread_create)
       return -1;
   
   
   /* Wrap user function. */
   warg = malloc(sizeof(struct wrapper_arg));
   if (!warg)
       return -1;
   warg-&gt;fn = start_routine;
   warg-&gt;arg = arg;
   
   
   /* Create a thread with a wrapper. */
   return libc_pthread_create(thread, attr, &amp;thread_wrapper, warg);
   
   
   }
   

   Makefile

   CFLAGS+=-fpic -O3
   checker.so: checker.o
       gcc -shared -Wl,-soname,$@ -o $@ $^ -ldl -lpthread
   

Answer 11

A Tool Approach

You can use Valgrind to help with this (via it's Helgrind tool), but it requires minor modification of the code. For each thread, you make the thread lock a unique mutex when the thread is created, and release the mutex when the thread exits. Then, when run under Helgrind, you will get a warning if the thread hasn't exited when the program terminates because the thread will still be holding the lock to the mutex. Consider this example thread start routine:

void * thread_start (void *arg)
{
    pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;

    pthread_mutex_lock(&mutex);

    // ...
    // Here the thread does whatever it normally does
    // ...

    // Unlock the mutex before exiting
    pthread_mutex_unlock(&mutex);
}

Simply run the program using Valgrind's Helgrind tool like so:

$ valgrind --tool=helgrind ./<program-name>

If the thread didn't exit when the program terminated, then Helgrind produces a warning like this:

==2203== Thread #2 was created
==2203==    at 0x31C96D3CDE: clone (in /lib64/libc-2.5.so)
==2203==    by 0x31CA206D87: pthread_create@@GLIBC_2.2.5 (in /lib64/libpthread-2.5.so)
==2203==    by 0x4A0B206: pthread_create_WRK (hg_intercepts.c:229)
==2203==    by 0x4A0B2AD: pthread_create@* (hg_intercepts.c:256)
==2203==    by 0x40060A: main (main.c:26)
==2203== 
==2203== Thread #2: Exiting thread still holds 1 lock
==2203==    at 0x4005DD: thread_start (main.c:13)
==2203==    by 0x4A0B330: mythread_wrapper (hg_intercepts.c:201)
==2203==    by 0x31CA20673C: start_thread (in /lib64/libpthread-2.5.so)
==2203==    by 0x31C96D3D1C: clone (in /lib64/libc-2.5.so)

You will get false positives using this method if you don't add the mutex unlock code anywhere the thread may exit (e.g. using pthread_exit), but fixing such a false-positive is easy once it is identified.

An Alternative Approach (Recommended)

Having said all of the above, that's probably not the approach I myself would take. Instead, I would write the program such that it cannot terminate until all threads have exited. The simplest way to achieve this is to call pthread_exit from the main thread before returning from main. Doing so will mean that the process will stay alive so long as any other thread is still running.

If you take this approach, and the process doesn't quit when you expect it to, then you know that a thread is still running. You can then attach a debugger to the process to determine which threads are still running and what they are doing.

Answer 12

Such tools already exists. On Linux you can use ps or top. On Windows, good ole Task Manager does the job:. Just check whether your process still exists:

• if the process still exists, it means that one or more threads in it are running.
• if there are no more threads running, the process is terminated.

Answer 13

You don't need any external tool for this: i would track the threads by using a simple semaphore instead.

1) set it up so that its initial count is the same as the number of your threads:

sem_init( &semThreadCount, 0, threadCount );

2) Modify your threads to "notify" they are exiting gracefully:

sem_wait( &semThreadCount );

3) You can either quit whenever the threads are finished or when the semaphore is 0, or just print the remaining semaphore value and quit, that will be the number of still-running threads:

int v;
sem_getvalue( &semThreadCount, &v );

This way you can both ensure no threads are still running if your exit or, with some logging, being able to know which ones are still running after you quit.

Remember to sem_destroy the sempahore as well.