tags:

views:

7

answers:

1
Q: 

getting the partition number in a reducer to which a key-value pair belongs

Hello everyone, when i am processing a given key-{set of values} pair in reducer function, how can I get the partition number to which this key-{set of values} belong to? How is it possible to get this partition number without adding extra information about the partition number with each key-value pair during partitioning? Cheers

A: 

This has worked for me: jobconf.getInt( "mapred.task.partition", 0); in the reducer.

 2010-07-08 09:38:00

related questions

How is MapReduce a good method to analyse http server logs ?
How do I control output files name and content of an Hadoop streaming job?
Parallelizing Ruby reducers in Hadoop?
CouchDB Views: created_at greater than a passed value
Mappers, Reducers, FIlters
Implementing large scale log file analytics
Hadoop: map/reduce from HDFS
What is the use of the 'key K1' in the org.apache.hadoop.mapred.Mapper ?
What programming language is Google written in?
What type of problems can mapreduce solve?
How to easily apply a function to a collection in C++
Implementing user ratings / favorites on CouchDB
Where to look for contributors?
Hadoop on windows server
CouchDB - .NET or Mono Equivalent Technology
.NET MapReduce Implementation
Map and Reduce in .NET
What is Map/Reduce
how to implement eigenvalue calculation with MapReduce/Hadoop?
20 Billion Rows/Month - Hbase / Hive / Greenplum / What ?
Large data - storage and query
Is there a .Net equivalent to Apache Hadoop?
How do you use MapReduce/Hadoop?
RT parallel processing in Rails
How do I tell a multi-core / multi-CPU machine to process function calls in a loop in parallel?