I have a For-loop which takes a very long time to execute,I am creating a new row in loop and adding it to data table.
what i want is handle this long running process on multiple threads.For eg :- for 2000 lines have one for loop on one thread,another 2000 lines on another thread and so on.
Please provide some source code to achieve this in c#.
Spawning an action happening sequentially, such as a loop, to a separate thread is not wise.
You will not get an increase in performance. Clearly you have some sort of bottleneck (the DB, if I understand correctly), and spawning several threads on that will not enhance performance. It will even worsen it.
Where is the time being spent? If in insertion into the DB then hitting the DB harder will probably make this go slower!
It really depends on what your code is doing in the loop. Is each iteration of the loop an embarrassingly parallel workload? Without seeing code it is impossible to tell if what you want to do it feasible or not since not every loop is parallelizable.
Its important to profile where the delay is actually coming from... I suspect its not the speed of your program but its the database thats taking time... multithreading wont help here
I'm not sure you'll benefit from splitting up your operations on several threads: you have one shared resource, which is not thread-safe for write operations (DataTable.Rows.Add is indeed a write operation).
If each data row takes very long to create, you can split creation of DataRow objects, push them into buffer and only then add these to a DataTable.
I'm not so sure why everyone seems to be down on the idea of you doing this. There are many cases that processes such as this can be substantially sped up through the use of multi threading. Some things to take into consideration though:
What is slowing the process down right now? Is the CPU the bottleneck and some cores are left unused? If so it may be a good target to make parallel, if it is disk, network or memory then you won't gain anything from splitting it across threads.
Does the order matter? Making sure that things finish in a certain order can be a pain in multi threading scenarios. If you need the results to come back in the same order as the for loop runs then you may need to figure out someway to sort them after everything has completed. With the extra processing this involves you may not gain anything.
Are there any shared resources? Will multiple threads be trying to access the same object? More importantly will they be trying to edit it (like a counter for instance)? You'll need to use locks in this case and all the time spent waiting for access may make a multi threaded solution slower than a single threaded one (or at least block more CPU resources).
Given all of these caveats though, you may find enormous benefit from a good parallel implementation. There are some libraries around which give you multithread versions of a foreach loop with very little work on your part. For example there is a Microsoft version here.