views:

12

answers:

0

I have, essentially a series of reduce jobs I am running on a lot of data using Hadoop Streaming. I am not really using my Mappers for anything, so am just using Identity Mappers, but I do need the default partitioner hadoop is giving me to group my data in a different manner for each step of my MR job.. I don't know enough the system well enough to know whether I can ask Hadoop to run the paritioner from the reduce step instead and avoid unnecessarily moving the data to a mapper and then to a new reducer. Is this possible?