Handling long page execution times in ASP.NET

Question

I'm working on a web application import program. Currently an admin user can upload a formatted csv file that my page will parse. I'm experiencing an execution duration issue as each line pertains to a file that has to be saved to Scribd, S3, as well as some internal processing.

What would you guys recommend for improving execution time? Since this is an admin only page, I doubt it would get run more than once a week, so my hope is to get it out the door asap.

I've looked some at the Async="true" flag, but I wasn't sure if that was the direction I wanted to go, or if I should look more that a windows server.

Answer 1

A quick and dirty option might be to set Page.Server.ScriptTimeout to a really high value on that page. (I think it maxes at Int.MaxValue).

Probably advisable to block the submit button after its been clicked, and inform the user that they may want to go make a coffee.

Answer 2

Two options come to mind:

Threads: In your code setup a collection of threads, join them and then have each one process a single file. Once all the threads complete you'll be able to return the page. This will increase your turn around time, but could still leave something to be desired on page returns

Queue: Have the user submit the csv file and provide a GUID/Hash/Whatever ID where the admin could then go to the "status" page, input their ID and check the details of their job. This solution will provide a quick feedback to the user and allow them to keep track of the results without having to wait around.

Answer 3

I'd suggest using AJAX to have an internal post back occur that would handle the asynchronous processing. You can periodically poll the state, and prevent your master page from having the "processing" wheel constantly churning on the page for the lengthy process.

Answer 4

I have a web page that takes a long time to process a mailing list so I kick it off in it's own thread. When the process is done, a report can be seen from another link on the result page. I have a runable MailSender class. The ASPX script has a bit in it that looks like this:

// prep the MailSender
MailSender ms = new MailSender(people, Subject, FileName....);                       
if (SendAsync) {
    ThreadStart ts = new ThreadStart(ms.run);
    Thread WorkerThread = new Thread(ts);
    WorkerThread.Start();
} else {
    ms.run();
}

If you want to speed your code up, try to break it into parallelizable pieces if you can and write a class for each piece. You could then kick off a new thread for each bit and monitor the status somewhere so the user can be informed when to come back to the results. You said that each line of your input would generate it's own output file. Sounds like a great candidate for multi-threading. Won't speed things up much if you don't have multi-cores availabe on the server though.

One problem with this whole scheme is that server restarts or application pool recycling will kill your long running process. This can be a problem if you threads are going to run for an hour or two.

Answer 5

As external factors are involved in the processing time, you need to consider if performance improvements would affect "actual" performance, if most of the time is in processing it and sending it to the thirdparty (ie Scribd,S3), then making improvements on your end might not have a huge affect and might increase the complexity for a simple task.

What I would do is have the aspx page only doing what aspx does best; ie handling the user interface part only (ie the upload), so once the upload is complete as far as the user is concerned their part is done. You could implement a progress indicator using AJAX to make it nicer but as its an admin section I wouldnt bother with the niceties,

Then have simple console application sheduled to fire at specific intervals, or a windows service watching a directory (depending on how timecritical the updates are), once the app runs as it is in the back ground and does not require user interaction, time is not a critical factor (ie you dont have a user waiting for context to be returned)..

it will appear to the user that things are very snappy (ie the time it takes to upload the file) and you are keeping needless complexity out of your solution.

Answer 6

I think the simplest solution to what you want is to use asynchronous pages in ASP.NET. Is there any particular reason why you don't want to go that route?

I can think of an alternative, which is to have some background process (like a process triggered by a scheduled task in Windows, or a Windows service) that will look at a queue of waiting jobs (say, from a database table) and process those jobs. This way you will have to upload that CSV somewhere and insert a db record so that the background process will see that CSV and use it when it comes around. But to me it seems like more work, so I'd rather use asynchronous pages :)

Here's a nice tutorial on ASP.NET asynchronous pages

Answer 7

We can actually handle long running process with some simple logic and code. check out following link, i found it very useful for handling long running processes over web.

http://aspdotnetmatters.blogspot.com/2010/08/aspnet-handling-long-running-process-in.html