I'm tossing around a few ideas for travel search engines and I'm wondering how these sites get their source data. Do they scrape all the content from airline homepages? This seems like an enormous job given the number of airlines etc out there.
Is there some API or web service standard that every airline conforms too?
Am I going to have to negotiate access to the data APIs for every airline I wish to index?
I would assume that the airlines expose all their data as feeds or web services - after all it's in their interest to make it as easy as possible for others to sell their flights.
Interesting question though!
The aggregators don't scrape the information. Nor do they need to.
The airlines supply the seat and flight info for the aggregators to sell on their behalf. It's often in the best interest of the airlines to work with these websites as they can bring in the eyeballs and in turn a higher churn and sell rate to help offload the seats.
Airlines are known to also sell directly from their own sites. And it's the same information the aggregators are given access to, but the airlines themselves sometimes sell at a slightly cheaper seat price.
They do use APIs, but not sure if it's to a standard format.
In the european case, there is an affiliate platform called tradedoubler that seems to provide such information through an API.
This answer has some useful links:
http://stackoverflow.com/questions/527007/airline-reservation-systems/527200#527200
You could also have a look at Amadeus and ITA websites(ITA was recently bought by google), but they are extremely expensive.
I would expect big news coming in this field in 1 year or so when google deal is approved.
@Fgblanch: The linked site doesn't seen to provide any such API, they are some kind of online advertising agency.