Long Running Workflows in Sharepoint, do they block the w3wp process

Question

We have a WSS 3.0 installation with Search Server, which is used to search for documents and Save the search definition to repeat the search later. The users want the option to be able to download all the files in their search results as a one-off Zip file.

I have a very basic solution where the Zipping of the files is done in the web part when the user clicks on the button, but if the zip file takes a while to create the user is left waiting (and I suspect, any other users accessing the site will be waiting because I imagine the compression of the documents is being done by the w3wp process).

I thought perhaps I could kick off the zip file creation as a workflow instead, and the user be allowed to download the file once the workflow is complete, but I've now realised that workflows run under the w3wp process too.

If a workflow task is taking a long time to execute (if for example the user had picked a large number of documents to download), would it impact other users of the sharepoint site and stop them accessing any pages until the workflow has completed?

Obviously we are going to place some limitation on the maximum size of the documents the user can zip up to download so that we don't kill the machine, but I'm still worried that whatever limit we place, the workflow process could still end up locking out other users. Is this the case? Are there any better suggestions for creating such a task which would not affect other users?

Thanks

Answer 1

Put a delay activity in the workflow before the activity that does the ZIP creation. This will push the workflow from the interactive W3WP process to the WSSTimerV3 service since it needs to run at a future time.

Regards, Paul

http://blogs.msdn.com/pandrew

Answer 2

Even when you were doing the zip in the web part, you were not blocking other users. The w3wp worker thread that processed the request was blocked waiting for the zip to complete, but all the other worker threads were able to continue.

Still, this could become a scalability issue if there were many waiting-for-zip threads: eventually, incoming requests might have blocked waiting for worker threads to become available. That's a reason to use asynchronous processing in ASP.NET.

Using a workflow will have helped, because the workflow was kicked off, and the request completed, permitting other requests to be processed.

You were concerned about the workflow running in w3wp. However, I don't know that it was running on one of the worker threads within w3wp. I don't know how SharePoint configures its workflow host, but I suspect that it uses a different set of threads.

You might want to do some load testing of this to find out. Create a dummy web part that just runs a zip as soon as you request the page containing it. Run up a load to that page and find out how many requests you can get before they begin to queue up waiting for worker threads to become available. Then do the same thing, but have the web part kick off the workflow. Again, see how many requests you can run at the same time before requests start to queue up.

Answer 3

1. If the zipping takes less than 5 seconds I would just do it synchronously in the same thread and be done with it. Least complexity, best user experience, no blocking of other users (limited by the ASP.NET thread pool size). A bunch of clicks will kill the server though.

2. If you have large files or lots of traffic you could persist a First In First Out queue in a database and have a Windows service take them out and execute them. This way you have control over the thread count used to zip files. This solution gives you O(1) algorithmic complexity but greatly increases the complexity of the design. You may want to consider using something like AJAX to tell the user "You are 4 of 45 in the queue...".

3. If you have a large variation of file sizes you may want to implement the top two solutions as strategies and implement a third adaptive strategy that defers to one of the previously mentioned strategies by looking at things like unzipped file size and server resource availability. Good compromise between user experience and resource availability but the most complex (expensive).

Groete, Hans