How to Background time/IO-consuming operations across a firewall in ASP.NET C#

Question

Summary
We have an ASP.NET application that allows users to query a SQL Server Database through a middle layer (pretty standard stuff). When the user attempts to generate a PDF for the rows that were returned to them. Each set of rows with a common identifier will "turn into" a separate page in the resulting PDF.

Originally, I assumed that the number of rows would be of an order of magnitude that was conducive to piping the end PDF into the ASP Response stream and letting the user download the file to their machine, but newer requirements indicate that the number of rows is very large and vastly outside of the scale for which I had designed this.

My current solution is as follows:

1. If the returned row count is larger than a configured threshold value, the user gets a message that asks whether or not they want to generate the PDF's in the background and be notified by email with a URL when its done, so they can download them.

2. If acceptable to the user, I create two DataTables: One with the actual data, and one with MetaData such as where to save the resulting PDF, the name, and other meta-type information.

3. I can then use DataTable.WriteXML to generate a "job" file in a particular folder.

4. This is all done in a separate Thread, so the User can continue doing what they want.

Problem:
I really want to store the job xml file on a machine inside our firewall (shared network folder), but It seems I cannot for security reasons. I have been told I might want to use Impersonation and have tried but could not get it to work.

The other part of this application is something that "listens in" to the "input" folder, and processes these "job" files once every hour (or similar schedule), moves the generated PDF file to the requisite location and emails the user.

I have gotten the part that reads job files and generates the PDF working, and the part that creates the job files works (when I run it inside the firewall), but when I run it outside the firewall, even attempting impersonation, it does not work. (keeps saying "Network location not found".

Question(s):
Am I going about this the right way?
How do I write these files across the firewall?
Any design ideas or suggestions?

Thanks in advance.

Answer 1

Can you at least do http across the firewall? Then don't create an XML file, but call a webservice that is inside the firewall. Idealy, this web service would not create an XML file either, but instead put the data in a messagequeue. MSMQ Could be useful here. The PDF generator would then read the XML off this queue. This way you won't run into the situation where the generator picks up a file that is not complete yet. And you don't waste time and resources polling for files.

Hope this helps.