AI testing framework

Question

I am looking at developing an AI player for a simple game I have created in C#. I will be creating a population of the bots and evolving them over generations.

What I was wondering is there any frameworks out there that could be good for this sort of testing / development. Ideally I would like something that I could plug any / some type of games into and say, OK so have a population of X run it over Y generations and chart the results for me.

I was having a think about how I would create something that would do this for me and allow me to reuse this later for different AI projects and all I could think of was to have some sort of core code and some interface contracts that the game and AI would use so that the server can script it.

What are your thoughts, does anyone have any practical experience of this sort of thing?

Answer 1

I'm not experienced with AI, but personally I'd start with a standard unit testing framework. You can extend your test classes (or even extend many frameworks) with helper methods as needed.

Flipping to the appendix in Osherove's The Art of Unit Testing, I'd start with NUnit and try MbUnit or xUnit if it didn't meet my needs.

And I echo Ravadre's comments above.

EDIT: I reread your original post, and I'm not sure that my answer was sufficient. While I highly advocate unit testing (or better yet, TDD) during development, you seem to be looking for a way to tell which bots are most successful. This goes beyond unit testing and (arguably) beyond integration testing. My (untested) suspicion is that it wouldn't be exceptionally difficult to write a harness for your code from scratch that would do exactly what you want - run multiple iterations/permutations and chart the results. Test frameworks are going to give you pass/fail results, but that's not all you want. If you create interfaces for your bots and games, you could easily plug them in.

Answer 2

It is called genetic algorithm framework. Any flavor is available through google.