What is the best way to "cache" frequently changing search results

views:

answers:

+1 Q:

What is the best way to "cache" frequently changing search results

I'm am currently developing an ASP.NET intranet application. One of the features of this application allows users to search for items in order to manage them (edit, delete etc).

Currently I am not using any kind of caching (since the search results change so frequently); however, when the user's search results in around 4 thousand items, it takes the system almost 13 seconds to return. Because the search results are not being stored between page requests, it is going to take 13 seconds for the page to process for any postback done on the page (sorting, paging, ...anything) .

I have been doing research on ASP.NET caching but I don't think that it is really meant for the purpose of caching results that are frequently going to be changing.

What I'd like to know is what is the best way to cache/store my search results?

Session? (It's already getting bloated with other data...)

Thanks for sharing your thoughts,

-Frinny

You don't want to show invalid data. Your best bet is to stick with paging and use AJAX to make quick hits to the server to grab the next page.

ChaosPandion 2010-01-29 15:08:15

Thanks for your reply ChaosPandion but how is Ajax going to help speed up anything? Even using Ajax to make the paging look smoother, it's still going to take 13 seconds to process the request....

Frinavale 2010-01-29 15:38:06

+3 A:

Sounds more like you have two problems.

Taking 13 seconds to return a result is way too long. You need to optimize that query and get it down to a sub-second response.

Secondly, you're returning too many items. Nobody will page through 4,000 items, if they are searching. Instead, if they get that large of a resultset, they should be refining their search criteria.

Chad 2010-01-29 15:14:08

This is what I should have said. :)

ChaosPandion 2010-01-29 15:15:14

+1. Yes, 13 seconds is way to long even accounting for paging, sorting, etc. Look into different strategies on how to do this. Hint: your web code is NOT the right place. Time to get dirty with SQL.

Chris Lively 2010-01-29 15:18:46

one more thing, we have tables with hundreds of thousands of records. Searching, paging, and sorting is all handled via a single s'proc call. Our page response time is somewhere between the act of pushing the mouse button down and letting go of it.

Chris Lively 2010-01-29 15:20:29

I totally agree that 13 seconds is too long. I do not have control over the query though. This is done by a third party SDK and that SDK takes 13 seconds to return 4000 records...I have many fields that let the user refine their search criteria. Hopefully they are smart enough to know how to search for the exact items that they want to work with (I know I would....but the average user?)

Frinavale 2010-01-29 15:40:58

@Frinivale, sounds like you need a new 3rd party tool, you're outgrown the one you have. Can you get at the data another way? Perhaps, run a service that uses the SDK to keep a sql db in sych that you can query against? That, or if your users can handle a 13 second delay you should store their search results (all 4000 records) with their filter as the key until they make a new search. That way they can query, take the speed hit once, then page through it quickly.

Chad 2010-01-29 15:54:30

Sadly there is no other way to get the data. I'm not about to start packing a database with my application when it's deployed just to solve this 13 second "what if"....are you saying that caching this data is not a good idea then?

Frinavale 2010-01-29 21:48:24

Well, not for long if it's frequently changing. I would cache the result of the query for the length of the users action. In short, cache the result of the last search so you can page on it quickly, or filter it further quickly. As an example, we have one app the user gives a first set of conditions, they get a huge resultset that takes a few seconds to retrieve. They then can page, sort, and filter further on that resultset without going back to the db.

Chad 2010-01-29 22:23:34

I was originally caching the results (well, storing it into session) but some requirements changed which now specify that the user should be able to open several tabs and work with the search results of that particular tab. Session is going to grow way too large if I keep using it and I'm afraid it will cause the application to recycle. So I started to consider using Cache instead. I've never used Cache before, but from what I've been reading I don't think that is really meant for this purposes...at this point I'm going to give it a try and see what happens...

Frinavale 2010-02-01 16:05:52

caching frequently changing data for me too doesn't seem to sound good,use proper indexing and paging to low down the overhead.

Vinay Pandey 2010-01-29 15:15:44

ansaurus

tags:

views:

answers:

What is the best way to "cache" frequently changing search results

related questions