The virtual memory available to a process is 2Gb of the 4Gb address space. Each Thread reserves about 1Mb of virtual memory space by default for its stack space. win32 applications therefore have a limit of about 2000 live threads before virtual memory becomes exhausted.
Virtual memory is the memory that applications get in modern virtual memory OS's like windows. What happens on Win32 is, your application gets given a 2Gb virtual address space. When your program calls new or malloc, after tunneling trhough several layers, space gets allocated for your application ON DISK - in the pagefile. When CPU instructions try to access that memory, hardware exceptions get thrown and the kernel allocates physical RAM to that area, and reads the contents from the pagefile.
So, regardless of the physical RAM in the PC, each and every application believes it has access to a whole 2Gb.
Virtual Memory is a count of how much of your 2Gb space has been used up.
Each thread (see above) reserves 1 Mb of virtual address space for its stack to grow. Most of that 1Mb is just reserved space (hopefully) without the backing of RAM or pagefile.
When you close a thread handle you do NOT close the thread. threads are terminated by another thread calling TerminateThread (which leaks the threads stack and some other resources so NEVER use it), calling ExitThread() themselves, or by exiting from their ThreadProc.
So, with the 2000 call limit, the unmatched CoInitialize and CoUninitialize calls, I would say that your threads are not exiting cleanly or at all. Each of the 2000 worker threads is stuck doing something rather than exiting after finishing their work.