Consumer profiles with analytical scores [ConsumerID, 1..n demographical variables, 1...n analytical scores e.g. "likely to churn" "likely to buy an item > 100$ in worth" etc.] have to be possible to query fast if they are to be used in customizing web-sites, consumer communications etc.
Well. If you have:
- Large number of consumers
- Large profiles with a huge set of variables (as profiles describing human behaviour are likely to be..)
...you are in trouble. If you really have a physical relational database to which you target a query and then a physical disk starts to rotate someplace to give you an individual profile or a set of profiles, the profile user (a web site customizing a page, a recommendation engine making a recommendation..) has died of boredom before getting any observable results.
There is the possibility of having the profiles in memory, which would of course increase the performance hugely. What are the most proven solutions for a fast-response, scalable consumer profile storage? Is there a shootout of these someplace?