YUI Datatable failing in IE for large datasets

Question

I have a DataTable and DataSource (YUI 2.6). The XHRDataSource connects to an XML producing address which is a servlet, where I write the XML out onto the response via the PrintWriter.

Servlet:

String data = dataProvider.fetch(request.getPathInfo());
int cLen = data.length();
response.getWriter().append(data); 
response.setContentLength(cLen);
response.setContentType("text/xml");
response.getWriter().flush();

javascript:

var url = "../data/SomeProvider";
this.myDataSource = new YAHOO.util.XHRDataSource(url);
this.myDataSource.responseType = YAHOO.util.DataSource.TYPE_XML;
this.myDataSource.connXhrMode = "queueRequests";
this.myDataSource.responseSchema = responseSchema;
this.myDataSource.maxCacheEntries = 0;

It works in FF3 fine. I can see via Firebug the xml getting returned, it looks good; the table and everything else hooked to the data source render fine.

In IE8, it fails for the full dataset (390 rows .. not that big, really) and the data table claims no rows were found. However, if I reduce the size down (to say, 20-30 rows) IE works fine. I have been searching high and low but I'm out of ideas - any clue what I'm missing?

EDIT Additional information. The failure is right when the XML response crosses the 8192 character mark. From what I've read, IE has a limit of 8192 characters in the URL or the parameter string - but why would that limit apply to data written into the response stream itself? Or do XMLHttpRequests get handled differently?

Answer 1

I figured it out, but I have no idea why it is so.

adding:

response.setBufferSize(cLen);

to the servlet makes IE happy. I guess that parameter defaults to 8192 and IE doesn't ask for the rest of the stream? Like I said, I don't know why it works. Which makes me nervous!