C# Console App Threading?

Question

Hi,

I'm about to write a console app which will run once a week at 2am on Sunday Morning. It's going to interrogate a SQL server database, then perform some calculations and then write the results to a new database.

It's processing hundreds of thousands of records so it's going to take hours to complete.

I'm going to create a C# console app and use windows scheduler to start it.

My questions are:

1) Do you think this is a good approach to take? 2) In the past, similar console apps have put the CPU usage at 100% while it continuously loops. Is there a good way to properly handle threading etc?

Any advice before I start would be very much appreciated.

T

Answer 1

In general, its a good approach because it schedules CPU & Data intensive process on non-peak/off business hours so user experience and application availability will not get diminished. Only consideration would be - how much total time your worker process might be taking? If its going to run into 30-40 hours then it may affect business hours on Monday.

As far as threading goes, multiple threads will help only if your calculations take some time and not happening on database server. So in such case, while one thread is waiting on data from database other thread may consume CPU. However, multi-threading means, you should able to divide your data horizontally so that each thread can work on different set of records.

Answer 2

If you are worried that your app might continue to run when it actually interferes with other users/processes you could just start the task in low priority. That way when other CPU intensive processes run yours will take a back seat.

As far as multithreading goes, it depends on your situation. If you have multiple cores/cpu's to spare, and if you are able to divide your problem in parallel tasks, then you might be able to take advantage of more than 1 thread. At the same time, if you have multiple cores/cpu's to spare then you also have a smaller risk of hogging up all the CPU with a single threaded application since other processes will just be redirected to use other cores/cpu's.

I have used your strategy of scheduling a console app to do dirty work on off peak times. I haven't had any issues so far, and I haven't gone as far as to split tasks and go multithreaded.

Answer 3

You might look at the Tasks.Parallel classes (more info here). I recently implemented something similar and came across the same issues. Lucky for me, its a dedicated server to handle this...