Does sending a dictionary through a multiprocessing.queue mutate it somehow?

Question

I have a setup where I send a dictionary through a multiprocessing.queue and do some stuff with it. I was getting an odd "dictionary size changed while iterating over it" error when I wasn't changing anything in the dictionary. Here's the traceback, although it's not terribly helpful:

Traceback (most recent call last):
  File "/usr/lib/python2.6/multiprocessing/queues.py", line 242, in _feed
    send(obj)
RuntimeError: dictionary changed size during iteration

So I tried changing the dictionary to an immutable dictionary to see where it was getting altered. Here's the traceback I got:

Traceback (most recent call last):
  File "/home/jason/src/interface_dev/jiva_interface/jiva_interface/delta.py", line 54, in main
    msg = self.recv()
  File "/home/jason/src/interface_dev/jiva_interface/jiva_interface/process/__init__.py", line 65, in recv
    return self.inqueue.get(timeout=timeout)
  File "/usr/lib/python2.6/multiprocessing/queues.py", line 91, in get
    res = self._recv()
  File "build/bdist.linux-i686/egg/pysistence/persistent_dict.py", line 22, in not_implemented_method
    raise NotImplementedError, 'Cannot set values in a PDict'
NotImplementedError: Cannot set values in a PDict

This is a bit odd, because as far as I can tell, I'm not doing anything other than getting it from the queue. Could someone shed some light on what's happening here?

Answer 1

There was a bug fixed quite recently where a garbage collection could change the size of a dictionary that contained weak references and that could trigger the "dictionary changed size during iteration" error. I don't know if that is your problem but the multiprocessing package does use weak references.

See http://bugs.python.org/issue7105