cuda device selection with multiple cpu threads.

Question

Hello.

Can you tell me how cuda runtime chooses GPU device if 2 or more host threads use cuda runtime?

does the runtime choose separate GPU devices for each thread?

does GPU device needs to be set explicitly?

Thanks

Answer 1

Yes the GPU device needs to be set explicitly or the default one would be used (device 0 usually)

Keep in mind that once the runtime starts using one device all the functions called in the same thread will be binded to that device.

something I find useful upon starting a thread is

cudaThreadExit(); // clears all the runtime state for the current thread
cudaSetDevice(deviceId); // explicit set the current device for the other calls
cudaMalloc
cudaMemcpy
etc.. 

The programming guide has a chapter dedicated to it.