Progressively stream the output of an ASP.NET page - or render a page outside of an HTTP request

Question

I have an ASP.NET 2.0 page with many repeating blocks, including a third-party server-side control (so it's not just plain HTML). Each is quite expensive to generate, in terms of both CPU and RAM. I'm currently using a standard Repeater control for this.

There are two problems with this simple approach:

1. The entire page must be rendered before any of it is returned to the client, so the user must wait a long time before they see any data. (I write progress messages using Response.Write, so there is feedback, but no actual results.)
2. The ASP.NET worker process must hold everything in memory at the same time. There is no inherent needs for this: once one block is processed it won't be changed, so it could be returned to the client and the memory could be freed.

I would like to somehow return these blocks to the client one at a time, as each is generated. I'm thinking of extracting the stuff inside the Repeater into a separate page and getting it repeatedly using AJAX, but there are some complications involved in that and I wonder if there is some simper approach. Ideally I'd like to keep it as one page (from the client's point of view), but return it incrementally.

Another way would be to do something similar, but on the server: still create a separate page, but have the server access it and then Response.Write() the HTML it gets to the response stream for the real client request. Is there a way to avoid an HTTP request here, though? Is there some ASP.NET method that would render a UserControl or a Page outside of an HTTP request and simply return the HTML to me as a string?

I'm open to other ideas on how to do this as well.

Answer 1

Another way would be to do something similar, but on the server: still create a separate page, but have the server access it and then Response.Write() the HTML it gets to the response stream for the real client request.

You could do this if you extract the contents of each repeater row into a UserControl. You could then load the control dynamically (if need be) using LoadControl method, give it the proper data it should render (call DataBind if necessary) then render it to the response stream using the Render method. This is basically what you had in mind without the mess of creating a whole new request.

Disclaimer: I don't know if this will perform better or worse than your current solution in terms of total execution time.

Answer 2

This way doesn't use AJAX:

<div><script src="pageparts.aspx?partid=news"/></div>
<div><script src="pageparts.aspx?partid=weather"/></div>
<div><script src="pageparts.aspx?partid=email"/></div>

The way this works: pageparts.aspx emits javascript that constructs the relevant part of the page. The parts won't appear in any predictable order. You'll probably want to fix the layout of the parts so the page doesn't jump around a lot as it fills in.

The result is something like iGoogle.com (which is why I used those kind of URLs).

Your server-side code would look like this:

<asp:Repeater id="rpParts" runat="server">
    <ItemTemplate>
        <div><asp:Literal id="litPart" runat="server"/></div>
    </ItemTemplate>
</asp:Repeater>

You generate the <script> tag in the codebehind. All the code that actually constructs the data and puts it inside a big document.write() goes in pageparts.aspx

This may not work if you're breaking up one big database query into many small ones, because it can end up taking much longer overall.

Edit

the entire issue is in "the code that actually constructs the data"! If I could get that as a string

This doesn't make a lot of sense to me. What are you currently sending to the client, and how are you building it?

Also, just for the sake of completeness, here's a purely non-javascript approach:

<div><iframe src="part1.aspx"/></div>
<div><iframe src="part2.aspx"/></div>
<div><iframe src="part3.aspx"/></div>

This is ugly, and usually looks it, will work even when you have no control of the data.

Answer 3

Making ajax calls to a webservice to get partial html results. You can use the ViewManager described here to render the web controls in your webservice and return the resulting html.

http://weblogs.asp.net/scottgu/archive/2006/10/22/Tip_2F00_Trick_3A00_-Cool-UI-Templating-Technique-to-use-with-ASP.NET-AJAX-for-non_2D00_UpdatePanel-scenarios.aspx