Design of the recommendation engine database?

Question

i am currently working on recommendation systems especially for audio files.but i am a beginner at this subject.i am trying to design database first with mysql but i cant decide how to do it.İt is basicly a system which users create profile then search for the music and system recommend them music similar to they liked.

1. which database should i use ?(Mysql comes my mind as a first guess)
2. it is a web project and also then with mobile side.Which technologies should i use?(php,android platform...)
3. what are the pitfalls of this project.
4. how to design database for system like that?

Answer 1

Well, this is a vague question and a half, but I'll do my best to answer:

1. MySQL is a solid database, and so is PostgreSQL. Both are free and open sourced. MySQL is more widely supported and a little easier to use, but Postgres has some very cool features and functionality that's worth taking a gander at. WikiVS has a good comparison of the two.
2. Smartphones are having better and better browsers. Use PHP or ASP.NET (whatever you're comfortable with), and then build out a mobile site which looks better on the smaller resolutions.
3. There are a lot. First and foremost, how good is your recommendation algorithm? Secondly, storing audio files can eat up storage space quickly. What's your plan for scaling? Thirdly, how well do you know database design? Can you design a large, hefty database and index it properly? If not, you need to start reading everything you can on indices and database design. Fourthly, it's a software project, and those always have pitfalls. The best you can do is post here when problems arise and we can always see what the fine people of StackOverflow can do to help.

Answer 2

Go read "Programming Collective Intelligence". They have a number of fine algorithms for recommendations in Chapter 2, "Making Recommendations".

Answer 3

Any relational database should be good for storing the raw data like lists of songs, list of users, users' song preferences..

I think that you'll find that a relational databases (and SQL) are not that great for storing the various data structures that your recommender will be constructing. Your recommendation engine will probably creating data that doesn't really need to be in tables and manipulating it for storage in a relational database may just be wasted work.

Just be aware of what you are doing and don't spend time putting stuff into a SQL database if it feels wrong. Maybe look into using a document oriented database like MongoDB.

The recommender that I recently wrote is actually a Java server process that reads in the raw data from MySQL, does all of its work in-memory, and provides recommendation data to my application via an HTTP API. I didn't even bother storing the recommendation data permanently since it can be regenerated.