MPI4Py Scatter sendbuf Argument Type?

Question

I'm having trouble with the Scatter function in the MPI4Py Python module. My assumption is that I should be able to pass it a single list for the sendbuffer. However, I'm getting a consistent error message when I do that, or indeed add the other two arguments, recvbuf and root:

  File "code/step3.py", line 682, in subbox_grid
    i = mpi_communicator.Scatter(station_range, station_data)
  File "Comm.pyx", line 427, in mpi4py.MPI.Comm.Scatter (src/
mpi4py_MPI.c:44993)
  File "message.pxi", line 321, in mpi4py.MPI._p_msg_cco.for_scatter
(src/mpi4py_MPI.c:14497)
  File "message.pxi", line 232, in mpi4py.MPI._p_msg_cco.for_cco_send
(src/mpi4py_MPI.c:13630)
  File "message.pxi", line 36, in mpi4py.MPI.message_simple (src/
mpi4py_MPI.c:11904)
ValueError: message: expecting 2 or 3 items

Here is the relevant code snipped, starting a few lines above 682 mentioned above.

for station in stations
        #snip--do some stuff with station
        station_data = []
        station_range = range(1,len(station))
        mpi_communicator = MPI.COMM_WORLD
        i = mpi_communicator.Scatter(station_range, nsm)
        #snip--do some stuff with station[i]
        nsm = combine(avg, wt, dnew, nf1, nl1, wti[i], wtm, station[i].id)
        station_data = mpi_communicator.Gather(station_range, nsm)

I've tried a number of combinations initializing station_range, but I must not be understanding the Scatter argument types properly.

Does a Python/MPI guru have a clarification this?

Answer 1

Jim, that's simultaneously smug and utterly useless.

Answer 2

If you want to move raw buffers (as with Gather), you provide a triplet [buffer, size, type]. Look at the demos for examples of this. If you want to send Python objects, you should use the higher level interface and call gather (note the lowercase) which uses pickle internally.