How to end a thread in java?

Question

hi all, I have 2 pools of threads

ioThreads = (ThreadPoolExecutor)Executors.newCachedThreadPool();

cpuThreads = (ThreadPoolExecutor)Executors.newFixedThreadPool(numCpus);

I have a simple web crawler that I want to create an iothread, pass it a url, it will then fetch the url and pass the contents over to a cpuThread to be processed and the ioThread will then fetch another url, etc...

At some point the IO thread will not have any new pages to crawl and I want to update my database that this session is complete. How can I best tell when the threads are all done processing and the program can be ended?

Answer 1

A typical way would be to use common (volatile or synchronized) boolean flag(s) to communicate between the threads. When the IO thread is finished, it flips the flag. The other thread checks the flag value in a loop, and when it sees the changed value, it exits the loop and terminates.

If you use the producer-consumer model with a work queue between the IO threads and the processing threads, another possibility would be to pass a special "end of processing" token to the queue, which would signal to the processors that they can terminate.

Answer 2

You can wait for all threads to finish using CyclicBarrier for example http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/CyclicBarrier.html

Answer 3

Lay out the program logic. Store URLs in a Stack object (stacks are thread-safe).

if

a. there are no more URLs on the stack
b. no more crawler threads running
c. no more CPU/processing threads running

Then program can write to DB and exit.