How to tell for which object attribute pickle fails?

Question

When you pickle an object that has some attributes which cannot be pickled it will fail with a generic error message like:

PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed

Is there any way to tell which attribute caused the exception? I am using Python 2.5.2.

Even if I understand in principle the cause of the problem (e.g. in the above example having an instance method) it can still be very hard to exactly pinpoint the propbem. In my case I already defined a custom __getstate__ method, but forgot about a critical attribute. This happened in a complicated structure of nested objects, so it took me a while to identify the bad attribute.

Edit: As requested here is one simple example were pickle intentionally fails (there are many more situations where this is the case).

import cPickle as pickle
import new

class Test(object):
    pass

def test_func(self):
    pass

test = Test()
pickle.dumps(test)
print "now with instancemethod..."
test.test_meth = new.instancemethod(test_func, test)
pickle.dumps(test)

This is the output:

now with instancemethod...
Traceback (most recent call last):
  File "/home/wilbert/develop/workspace/Playground/src/misc/picklefail.py", line 15, in <module>
    pickle.dumps(test)
  File "/home/wilbert/lib/python2.5/copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects

Note that there is no hint that the attribute test_meth causes the problem.

Answer 1

You could file a bug against Python for not including more helpful error messages. In the meantime, modify the _reduce_ex() function in copy_reg.py.

if base is self.__class__:
    print self # new   
    raise TypeError, "can't pickle %s objects" % base.__name__

Output:

<bound method ?.test_func of <__main__.Test object at 0xb7f4230c>>
Traceback (most recent call last):
  File "nopickle.py", line 14, in ?
    pickle.dumps(test)
  File "/usr/lib/python2.4/copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects

Answer 2

Got me to my problem in 10 seconds following the change. Thanks!