views:

545

answers:

3

I am using a Session variable to pass a datatable from 1 page to another. Sometimes the datatable can contain well over 100,000 records. After running it a few times, I get thrown a Out of Memory exception, so I guess I have a few questions?

Is Session the best way to handle this?

Does Session.Clear("session") release it from Memory? If not, does anything release the Session from memory?

If I store a datatable into a Session object and then I store another datatable into that same Session object, does it keep using up memory or does it write over the existing Session object?

A: 

use a app-server layer to hold the data and each page should take it from there...

Dani
What is the app-server layer? Can you elaborate a bit, possibly with an example? Thanks
Xaisoft
+4  A: 

I'll assume you're talking about In-Process session state.

You aren't actually storing the DataTable itself in session. You are instead storing a reference to a DataTable. Thus when you create a new DataTable, and add that to session, you're simply overwriting the reference. You still have two DataTables somewhere in memory, until garbage collection cleans up any to which there there are no live references.

Remember that garbage collection in .net is non-deterministic. That is to say, setting an object to null does not immediately release the memory. It simply marks it, and at some point in the future, the garbage collector may see the dead object and release the memory associated with it.

You probably want to rethink your design if you're running out of memory. It might be better to have the second page refetch the data again, perhaps from a caching layer (possibly implemented on an application server as suggested by another poster) or perhaps from the database itself.

Winston Smith
What do you mean by In-Process? Would I set the Session["datatable"] to null to have the garbage collector at sometime release it or set the datatable to null. Also, I am not clear on what the application server is?
Xaisoft
A: 

My first question would be why you need to store the entire database in the Session or the Application? Here is a good article that goes over all of your options and it advises against storing large amounts of data in the Session or Application caches. What issue are you trying to resolve by doing this?

Edit:

Are you display all the data at once on that page? ex. scroll down through 10000 records. If so that doesn't sound very user friendly (Assumption). Have you considered paging that data? You could have a page of 50 records and n number of pages. That would make the data call a lot faster and then you could implement filters, sorting, etc.

J.13.L
My main issue is that when I fetch the data, there is so much that it takes a while, sometimes a couple minutes. I store it in Session so I can access it on another page, but this causes memory issues. If I fetch the data again, it will take more time, but some of the other posters have talked about fetching the data again.
Xaisoft