How to display content from other sites on my own page?

Question

I know this isn't a specific programming question, but I really need to know how this can be done. How does a website like this:

http://www.dogpile.com/

display search results from google and other search engines on it's own page. The only way I can think about doing something like this is by using iframes but of course then the content won't really be on my page.

Any help is appreciated.

Answer 1

Have a look at curl. There are plenty of examples of its usage on that page.

Answer 2

They are using the public api's for the different search engines and building their pages from the results.

See:

Google's Search API
Bing Search API
Yahoo! Search API

Answer 3

On the server side download the content of their page, make all the relative references absolute, add their head entries to yours, add their body to yours, hope you don't get caught stealing someone else's content.

Answer 4

You could use AJAX, and just consider the remote page as a webservice that returns HTML. That is, just innerHTML it directly into your DOM.

Answer 5

when someone requests a search:

1. perform that search for the various search engines on your server
2. extract the content with xpaths, regex, etc
3. and then display the results on your own webpage