Python Scrapy , how to define a pipeline for an item?

Question

I am using scrapy to craw diferent sites, for each site I have an Item (different information is extracted)

Well, for example I have a generic pipeline (most of information is the same) but now I am crawling some google search response and the pipeline must be different.

for example

GenericItem uses GenericPipeline

but the GoogleItem uses GoogleItemPipeline, but when the spider is crawling it tries to use GenericPipeline instead of GoogleItemPipeline....how I specify which pipeline Google spider must use?

thanks a lot!

Answer 1

Now only one way - check Item type in pipeline and process it or return "as is"

pipelines.py:

from grabbers.items import FeedItem

class StoreFeedPost(object):

    def process_item(self, domain, item):
        if isinstance(item, FeedItem):
            #process it...

        return item

items.py:

from scrapy.item import ScrapedItem

class FeedItem(ScrapedItem):
    pass