Pitfall of storing dataTable(10,000 rows) in a session variable?

Question

Consider my dataTable contains 10,000 rows and i want to know the pitfall of storing datatable in a session variable... I want to use it until a new row has been added...

What type of session mode should i use?

Answer 1

If you need the data through out your user session, then storing it in Session is good. But in your case you can go for either ViewState or Cache. You can store in ViewState is better.

ViewState["Table"] = ds.Tables[0];

to retrieve it,

DataSet ds=new DataSet(); ds = (DataSet)ViewState["Table"];

1. If a session variable is read before it has been assigned or if the current session times out, it will result in NullReferenceException as no null checking happens before reading from session variable.
2. There is no easy way to track the usage of session variables, their key names, their data types, the approximate memory being used per session, etc.

But do remember one thing putting DataSet inside the ViewState will increase the page loading.

**

The most efficient way to do what you ask is to convert the dataset that was returned by the DB into a Generic List and then to use the session in order to hold the List. List will spend less memory then Dataset in session.

**

Answer 2

Firstly, do you really need that much data stored on a session level. Could you possibly store a bit more on an application level (cache objects).

If you have a farm, that is more than one iis server, then you will have to use either a in memory cache, or a sql cache. With a large blob the in memory cache would be faster. If its on a single computer in memory on that server would work.

I would seriously think about refactoring the problem. Can you introduce paging or some other form of data window in order to reduce the amount of data?

Answer 3

With dataset that big, I really dont think you should store it in session or viewstate. Storing in SQL state will have huge overhead of read and write too. Storing in memory session will significantly affect your worker process memory usage. Storing in ViewState is even worse as it will make page loading extremely slow due to page size and time taken to encode and decode the viewstate.

If you really have to do it, of all choices I will go for in memory session and pay the price. Otherwise I think you should consider refactoring your code and not having to cache datatable of a that size and pull out data in a set/page as you need to. Alternatively if you have layered architecture, have your application code cache it and when a request is made the application code can extract and return the record/subset to the UI when needed.

Answer 4

Never do that, its not recommended.. It affects the performance if your server has low memory and busy processing times.

Before proceeding with this you need to consider

1. is your server got a available memory?
2. How busy your server is?
3. Whether the data you goin to put in session will be shared across among multiple user requests?

Why do you want to store that much data into an session.Probably you might be doing this for pagination (in a datagrid i assume), then you have to reconsider your design.

Cheers