Amazon MapReduce no reducer job

Question

Hi.

I am trying to create a mapper only job via AWS (a streaming job). The reducer field is required, so I am giving a dummy executable, and adding -jobconf mapred.map.tasks=0 to the Extra Args box. In the hadoop environment (version 0.20) I've installed, no reducer jobs will launch, but in AWS the dummy executable launches and fails.

How can run a no reducer/ mapper only jobs in AWS?

Thanks, Sagie

Answer 1

There is no way to create a mapper only job in amazon M/R. I am using the Identity reducer (org.apache.hadoop.mapred.lib.IdentityReducer) instead.

Answer 2

you can also use cat or NONE as reducer argument.