In ASP.NET, is it OK to store user-specific data in profile with a caching mechanism rather than in session?

Question

I have been reading articles about state management in ASP.NET for few days for my web application. In this application, one user can create/save/execute queries, and I need to keep each parameter of a query before the user can save the query.
These parameters are string values, but the global size for one user may exceed few megabytes. We plan to have our website running w/ ~100 users simultaneously.

In these conditions, I believe that it will not be good to store these values in session w/ in-proc mode.

We already implemented a ProfileProvider:SqlProfileProvider, using a caching mechanism and with AutoSave=true.

What is the best solution: storing these values in profile or in session but in an SQL database?

Answer 1

Depends on your server memory and how concerned you are about it. A database is always nice for this type of thing because you have better concurrency and more stable method for storing your data.

My recommendation is that you use a sliding expiration on the cached data, say 10-20 minutes or so. That way, you won't be consuming so many server resources because as users become inactive their session data will be evicted.

For an example of how to do sliding expiration caching (also referenced here), you can do this:

public static void AddToCache(string key, Object value, int slidingMinutesToExpire)
{
        if (slidingMinutesToExpire == 0)
        {
            HttpRuntime.Cache.Insert(key, value, null, System.Web.Caching.Cache.NoAbsoluteExpiration, System.Web.Caching.Cache.NoSlidingExpiration, System.Web.Caching.CacheItemPriority.NotRemovable, null);
        }
        else
        {
            HttpRuntime.Cache.Insert(key, value, null, System.Web.Caching.Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(slidingMinutesToExpire), System.Web.Caching.CacheItemPriority.NotRemovable, null);
        }
    }

Answer 2

1) One can keep session as Out Of Proc (on SQL Server), so the question is more about where to keep them: in memory or in database. If you keep in memory, then it will be destroyed eventually automatically. If you keep in database, then you can use this data across sessions. Decide whether you need these parameters to be saved across sessions (user can switch computers; user can open your application once in a while) - does he need these parameters to be kept for him?

2) Think about optimization - few megabytes of parameters per user?!? If that means that user can have hundreds of queries and they should be available next time user logs into the system, then keep all that in the database.

Answer 3

We've had some level of success persisting user details to disk to keep it out of memory. When you want something specific you then have to read it back into memory and this can be a bit slow, but it works.

For instance, you can serialize some of the objects and write them to files on the web server's hard drive. Name the files according the session ID, username, whatever, and then you can read/write from/to the files as and when you need it.

Answer 4

The best solution depends on whether you plan to grow beyond a single server. In a load balanced configuration, InProc session state won't work correctly (without using sticky sessions, which presents other problems). Out of proc session state, such as State Server or SQL Server, requires that your state information be serialized and deserialized for each page access. With multiple MB per user, that could be very slow.

Cache is another option, as suggested by Dave. One issue there might be whether the cache needs to be kept in sync from one web server to another.

The usual approach to this kind of problem is to store the data in the DB and in Cache, and to only retrieve it for those pages where it's really needed (unlike Session, which is read for every page). Then use SqlDependency or SqlCacheDependency to cache the data in a way that it can be updated on multiple servers if the DB changes.