Ajax issue: delay in getting data from web service using innerHTML, please guide

Question

I am working on an ajax application which will display about a million records in an html table. Web service returns records from server, I build a logn string by concatinating data and tags and than put this string using innerHTML (not using DOM for getting better performance).

For testing I have put 6000 recods in database (stored procedure takes about 4 seconds in completion of its execution).

While testing on local system (database and application on same machine) it took about 5 minutes to display the records in page. After deplying on web server it did not responde even for more time. It looks very low performance. I put records in a CSV file and its weight was less than 2 MB. I couldn't understand why string concatinations to build html table and putting string in innerHTML is taking such a huge time (if it is the issue). Requiment is to show about million records in web page but performance on just 6000 records is disappointing. I am not gettign what to do to increase performance.

Kindly guide me and help me.

Answer 1

You're trying to display a million records on a single page? No matter how you optimize your server code, that's a LOT of html to parse/render, especially if it's in a table.

Even using .innerHTML isn't going to "save" you any time. The rendering engine is still going to have to parse/style/render/position many millions of table rows/cells and you WILL have to wait while it's working.

If you absolutely HAVE to show all those records on a single page, try to break things up into manageable chunks. Have the AJAX call return (say) 100 records at a time, put those into the table, then fetch another 100 records, etc... At least that way you'll see the content of the page growing, rather than having to sit there and wait for 1,000,000 table rows to get displayed in a single shot.

A better option would be to do pageination, where only 100 records are shown at a time and you present a standard navigation with << first / prev / next / last >> buttons to swap through "pages" of data.

Answer 2

As Marc stated, you need pagination. See if this helps - http://stackoverflow.com/questions/446196/how-do-i-do-pagination-in-asp-net-mvc

In addition to this you could optimize the result by employing master-detail pattern - fetch only the summary of the record (master) and on some action in master, fetch details and display on the screen. This will reduce the size of data being transfered from the server.

Answer 3

I agree with Marc B and byte; pagination is the way to go but sometimes the code will be trickier depending on your particular situation.

Although I am not using PHP and not ASP.NET, I had a similar issue with a very large data set when attempting to load through AJAX. The real problem was the back end data query, because it was constructed from a SQL view and included distributed joins and lots of aggregation. Even showing a wait spinner dialog box did not make user experience any better :)

We chose AJAX as a way to provide a better user experience, but the initial page load was still incredibly annoying. I finally took a hybrid approach which rendered the initial page load with PHP (no AJAX.) All subsequent data loads (which were main partial loads consisting of one or more rows) were then managed through AJAX.