Searching for a song while using multiple API's

Question

I'm going to attempt to create an open project which compares the most common MP3 download providers.

This will require a user to enter a track/album/artist name i.e. Deadmau5 this will then pull the relevant prices from the API's.

I have a few questions that some of you may have encountered before:

1. Should I have one server side page that requests all the data and it is all loaded simultaneously. If so, how would you deal with timeouts or any other problems that may arise. Or should the page load, then each price get pulled in one by one (ajax). What are your experiences when running a comparison check?
2. The main feature will to compare prices, but how can I be sure that the products are the same. I was thinking running time, track numbers but I would still have to set one source as my primary.

I'm making this a wiki, please add and edit any issues that you can think of. Thanks for your help. Look out for a future blog!

Answer 1

I would check amazon first. they will give you a SKU (the barcode on the back of the album, I think amazon calls it an EAN) If the other providers use this, you can make sure they are looking at the right item.

I would cache all results into a database, and expire them after a reasonable time. This way when you get 100 requests for Britney Spears, you don't have to hammer the other sites and slow down your application.

You should also make sure you are multithreading whatever requests you are doing server side. Curl for instance allows you to pull multiple urls, and assigns a user defined callback. I'd have the callback send a some data so you can update your page with as the results come back. GETTUNES => curl callback returns some data for each url while connection is open that you parse it on the client side.