tags:

views:

99

answers:

3

hello, i have an queue of urls and i an processing it with some function i need to run some 10 thread to read 10 website at same time ,,,

boost::thread tl(boost::bind(&thread::start, crawl, url));

we can use for loop but is that is the only way to use to create dynamic thread....

Edited

 while(!q.empty())
                     {


                         boost::thread_group Tg;
                         for(int k=0;k<=1;k++)
                         {

                         Tg.create_thread(boost::bind(&threadMaster::start, mastercrawl, q.front()));
                         Tg.join_all();

                         }
                     }

this works as single thread and

 Tg.create_thread(boost::bind(&threadMaster::start, mastercrawl, "http://www.google.co.in"));
                                 Tg.create_thread(boost::bind(&threadMaster::start, mastercrawl, "http://www.yahoo.com"));

this works as multi thread how can i make it work my first program as multithread

+1  A: 

you might consider OpenMP if your compiler supports it (e.g. Visual C++ does). You don't have to create threads by yourself but rather

#include <omp.h>
omp_set_num_threads(10);
#pragma omp parallel 
{
    #pragma omp for
    for (int i=0;i < queueLength; i++)
    {
        //load url[i]
    }
}
Philipp
ya hai omp is great and i tried this sample #include <omp.h>`#include <stdio.h>int main() {#pragma omp parallelprintf("Hello from thread %d, nthreads %d\n", omp_get_thread_num(), omp_get_num_threads());}`it says `undefined reference to `omp_get_thread_num'`
raja
i use gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
raja
@raja: See if `libgomp.so` is in `/usr/lib/`. Or, just keep on using Boost and don't worry about such things.
Potatoswatter
@raja: this means you have to properly link against the static link library containing the omp implementation.
Philipp
the queue has 2 datas now ` while(!q.empty()) { boost::thread_group Tg; for(int k=0;k<=1;k++) { Tg.create_thread(boost::bind( Tg.join_all(); } }` this run as single thread but ` Tg.create_thread(boost::bind( Tg.create_thread(boost::bind( Tg.join_all();` works perfect i dont know y i am wrong
raja
@raja: maybe you can edit your original post and provide more code there, it's better to read then. Furthermore, I wouldn't mix the boost thread code with omp...
Philipp
+1  A: 

Without C++0x, you cannot put a thread in a container such as vector, so you will probably end up creating some kind of loop.

You can create an array of thread and fill it, though. The usual tricks for "eliminating" a loop apply.

{
    boost::thread threads[10]; // or *threads = new thread[10];
    std::fill_n( threads, 10, boost::bind(&thread::start, crawl, url) );
} // destroy array: destructors wait for threads to finish.
Potatoswatter
i am getting an problem now the second thread waits till the first thread finishes
raja
@raja: If they all consume the same work queue, they should all finish at about the same time. I can't give more specific advice without seeing more code.
Potatoswatter
the queue has 2 datas now
raja
@raja: My suggestion should be sufficient for that kind of work. You need to put the loop inside the thread function, rather than spawning and joining with each iteration. Access the queue from a critical section in that loop. The goal is to avoid joining any thread in the group until all the work is done.
Potatoswatter
can u suugest me an example or any documentation..
raja
can u suugest me an example or any documentation..
raja
@raja: I don't know a good source on multithreading in general… shouldn't be hard to find as it's a hot topic. In terms of Boost, you want to use `lock_guard` and `mutex`, also from the Threads library, to implement that critical section. Google should provide ample examples.
Potatoswatter
A: 

[Threadgroup example] "this works as single thread".

From what we see, it should not. However, if you have a highly-contended mutex shared amongst those threads, you will see that only the thread holding the mutex is doing work. That would explain the observed bahavior. In particular, for your example, you should not be holding a mutex while you're doing the actual HTTP GET in the thread.

MSalters