Hi folks,
I'm indexing a list of links, these links update quite often so I'm automating thumbnails for the sites.
For most sites it's easy, as I just grab the biggest image on the page hoping it describes the content.
But other times there are videos as main content of the page.
Does somebody have tips with dealing with this? That would be great!
Regarding the usage of Webkit to create screenshots I found this
wkhtmltopdf uses an embedded copy of the WebKit render engine (used in Safari, Chrome etc.) to save a webpage to PDF, including all images (no Flash video though I guess). That could be a starting point for a much more accurate thumbnail.
There exists (free and paid) services that do exactly what you need. I use shrink the web:
If the sites you are scraping from support oEmbed, you can let them do the heavy lifting for you. Search this PDF for "oEmbed" for a good start. http://bit.ly/adx4bi