views:

482

answers:

2

Duplicate

http://stackoverflow.com/questions/33978/find-out-how-much-memory-is-being-used-by-an-object-in-python

I am using ipython to run my code. I wonder if there is any module or command which allow me to check the memory usage of an object. For instance:

1> a = range(10000)
2> %memusage a
1MB

Something like %memusage and return the memory used by the object.

Thank you very much.

+2  A: 

UPDATE: Here is another, maybe more thorough recipe for estimating the size of a python object.

Here is a thread addressing a similar question

The solution proposed is to write your own... using some estimates of the known size of primitives, python's object overhead, and the sizes of built in container types.

Since the code is not that long, here is a direct copy of it:

def sizeof(obj):
    """APPROXIMATE memory taken by some Python objects in 
    the current 32-bit CPython implementation.

    Excludes the space used by items in containers; does not
    take into account overhead of memory allocation from the
    operating system, or over-allocation by lists and dicts.
    """
    T = type(obj)
    if T is int:
        kind = "fixed"
        container = False
        size = 4
    elif T is list or T is tuple:
        kind = "variable"
        container = True
        size = 4*len(obj)
    elif T is dict:
        kind = "variable"
        container = True
        size = 144
        if len(obj) > 8:
            size += 12*(len(obj)-8)
    elif T is str:
        kind = "variable"
        container = False
        size = len(obj) + 1
    else:
        raise TypeError("don't know about this kind of object")
    if kind == "fixed":
        overhead = 8
    else: # "variable"
        overhead = 12
    if container:
        garbage_collector = 8
    else:
        garbage_collector = 0
    malloc = 8 # in most cases
    size = size + overhead + garbage_collector + malloc
    # Round to nearest multiple of 8 bytes
    x = size % 8
    if x != 0:
        size += 8-x
        size = (size + 8)
    return size
John Mulder
+4  A: 

Unfortunately this is not possible, but there are a number of ways of approximating the answer:

  1. for very simple objects (e.g. ints, strings, floats, doubles) which are represented more or less as simple C-language types you can simply calculate the number of bytes as with John Mulder's solution above.

  2. For more complex objects a good approximation is to serialize the object to a string using cPickle.dumps. The length of the string is a good approximation of the amount of memory required to store an object.

There is one big snag with solution 2, which is that objects usually contain references to other objects. For example a dict contains string-keys and other objects as values. Those other objects might be shared. Since pickle always tries to do a complete serialization of the object it will always over-estimate the amount of memory required to store an object.

Salim Fadhley
But if you pickle a list containing all root objects you're interested in, there will be no overestimation.
Constantin
Thank you very much. But I wonder if pickle will do any compression or not.
i-freaker
No, pickle does not compress. It simply eliminates redundancy.
Salim Fadhley