Using gaming concepts to build user agents for market research purposes

Question

I work for a market research company in the online space. We have been spending all of our cycles for over a year and a half building the next big thing in this space with regards to profiling our respondents (over time) to better place them in available surveys . Something that one of our researcher's has asked us for many times (rightly so) is a tool that will prove the worth of this new profiling system and predict the outcome of tweaks to it's many algorithms and rules to show which version of a rule set has a better outcome.

The goal is to be able to take a sliver of our profiling system (a static slice of Q&A data for a given time - sex:male/female, drinks:coke/pepsi/mt.dew, income:etc.) and run user agents (artificially developed software robots or agents) through our profiling system to see what the interactive results would be. As the Q&A data would be the same, the user agents abilities to choose answers would be the same, and only the algorithms and rules behind how the profiler works would change - this theoretically would allow us to pre-determine the outcome of any changes to our system. This result would then allow us to proof changes before pushing the changes to our production system. The hope would be that we could more easily catch any errors before releasing to the wild. But this would also allow us to test changes to the logic to hunt for optimizations in the profiler.

My question: For someone like me (C#/.NET mostly) who has really only worked in the web application space, where do I look to get started in building user agents that are able to interact with an outside system such as my profiling system? I specifically need to know how to spin up 1000 (one thousand) agents and have them interact with my profiling system (over a given amount of time) by being able to answer the questions that are presented to them by the profiling system based on characteristics that are dynamically defined on the user agent at the time of initialization.

An example of this is that I need some black agents, some chinese agents, some male agents, some female agents, some old agents, some new agents, some religious agents, some that drink coke, etc. and all of them mixed together to most appropriately resemble the world. We already have the demographic break down of our population so we can easily spin up 10% black males, 60% white female stay at home mothers, and all the other representations of our population.

My first thoughts for creating a system like this was to use the power of my XBOX 360, and some well thought out agents that resemble a person from an object oriented world with some added characteristics to be able to intelligently answer some questions...and guess at others.

In speaking with my colleague, it was suggested that I use some of the artificial intelligence frameworks out there and a 1000 cpu graphics card (we have one already) to get some super wicked fast performance out of loads of user agents. Where each CPU is an agent...(something like this).

Is there anyone out there with experience in this sort of thing? Proofing problems with a fictitious model of the world?

Answer 1

You say "interact with an outside system" - what is the interface to this system, and how does a person use it? Is it over the web? If so, you're wasting your time thinking about GPU optimisations and the like since your performance bottleneck will be the network, even over a LAN. In such circumstances you may as well just run the agents sequentially. Even if you could effectively spawn 1000 agents simultaneously (perhaps across multiple machines), chances are high that you'll just cripple the target server in an accidental denial of service attack, so it's counterproductive. However if you have the ability to change that interface to allow direct interprocess communication, you could go back to considering the massive parallelism approach. But then 1000 is not a big number in computing terms. It's likely you'd spend more time making the algorithm run in parallel than you'd save by having it that way.

As for 'artificial intelligence frameworks', I don't think there is anything quite so vague that would help you. AI and intelligent agents is a massive field - the book Artificial Intelligence: A Modern Approach which is a standard introductory text on intelligent agents is over 1000 pages long and contains maybe 20 or 30 totally independent techniques, many of which could apply to your problem, many of which won't. If you can specify more clearly what tasks the agent has to perform, and which inputs it has on which to make those decisions, picking a decent technique becomes possible. In fact, it may turn out that your problem doesn't require AI at all, if you have a clear mapping between agent demographics and decision making - you just look up the answer to use from the table you made earlier. So it's important to work out what problem you're actually trying to solve first.