Do they use partnership and APIs, scrape the data or use public apis from all the job sites. Specially interested in how they obtain data from other job sites like monster.com and hotjobs.
Implementing a program to do similar stuff, all ideas welcome/
Well, most of the site, such as Monster.com, Hotjobs, CareerBuilder have RSS feeds for their jobs. What Indeed has to do is to pull in the data and parse it for the job content. It also has to make sure not to include duplicate jobs. That is very easy to do. Once the jobs are in the database, then all you'd need is a nice and simple interface for searching against the database for a particular jobs.
Here is my site which is called newjoblist.com. It is just a dead simple front-end and a very complex back end. Everything is 100% automated and hands-free. http://newjoblist.com/
Hope this helps