I'm running a local, single-system test using Qizmt of a simple MapReduce operation. At the end of the 'Map' phase I am calling:
output.Add(rKey, rValue);
This is called let's say a million times, and the keys are 1,2,3,4,5,6 etc - each unique (I'm just testing, after all). I've checked that this is happening as intended. It is. The function is called a million independent times and the values input are correct.
To my surprise, in the 'Reduce' phase only, say, a hundred or so keys are actually received (that is, the 'Reduce' function is called only a hundred or so times). These keys seem to picked almost arbitrarily from the list mapped with a pattern such as the following: 0, 383, 766, 2858, 3241, 5716, etc.
The set of keys which make it through to 'Reduce' are consistent. ie. I will get 0, 383, 766, etc consistently each run of the task. However, if I add a second system to the cluster, the set of keys which make it through is changed entirely!
The stage between 'Map' and 'Reduce' is a bit of a mystery to me - and I'm really not sure where to look since my Map function appears to be behaving exactly as intended (that is, calling 'output' a million times). Does anyone with more experience of parallel processing have any idea where I should be looking, or anything that might cause this behaviour?