views:

2322

answers:

4

I need to programmatically find out exactly how much memory a given Java Object is occupying including the memory occupied by the objects that it is referencing.

I can generate a memory heap dump and analyse the results using a tool. However it takes a considerable amount of time to generate a heap dump and for such a tool to read the dump to generate reports. Given the fact that I will probably need to do this several times, my work would be a lot more agile if I could throw in some code in my project that would give me this value "runtime".

Any ideas as to how I could best achieve this?

ps: Specifically I have a collection of objects of type javax.xml.transform.Templates

+1  A: 

You can investigate your object graph during runtime by using reflection.

flybywire
What about the overhead on instances? This may also differ depending on the runtime and system (think x64...).
Lucero
+3  A: 

You will need to use reflection for that. The resulting piece of code is too complicated for me to post here (although it will soon be available as part of a GPL toolkit I am building), but the main idea is:

  • An object header uses 8 bytes (for class pointer and reference count)
  • Each primitive field uses 1, 2, 4 or 8 bytes depending on the actual type
  • Each object reference (i.e. non-primitive) field uses 4 bytes (the reference, plus whatever the referenced object uses)

You need to treat arrays separately (8 bytes of header, 4 bytes of length field, 4*length bytes of table, plus whatever the objects inside are using). You need to process other types of objects iterating through the fields (and it's parents fields) using reflection.

You also need to keep a set of "seen" objects during the recursion, so as not to count multiple times objects referenced in several places.

Varkhan
For what it's worth, object header uses at least that much, but I think older JVMs use more. That's easy (enough) to verify, too, by simple test code (create a million objects, gc, sleep a bit, check out free memory -- not theoretically guaranteed to work, in practice does).
StaxMan
Also: 4 bytes is for 32-bit references. For 64-bit systems 8 bytes I think.
StaxMan
Well I was thinking that too for a while, but couldn't find any reference on that... so I tried some benchmarking as StaxMan described, and it seems it's still 4... go figure.
Varkhan
Maybe 64-bit VMs use compressed pointers... that would make sense.
Varkhan
Isn't it possible for different JVMs to be slightly different on some of this?
matt b
Yes, I think it is entirely possible. Which is why any such implementation is going to be a (hopefully good enough) approximate.
Varkhan
+2  A: 

A good generic solution is to use heap size delta. This involves minimal effort and is re-usable between any type of object / object graph. By instantiating and destroying your objects many times and garbage collecting in between, and then taking the average, you avoid compiler and JVM optimizations that alter results and get a fairly accurate result. If you need an EXACT answer down to the byte then this may not be the solution for you, but for all practical applications that I know of (profiling, memory requirement calcualtions) it works extremely well. The code below will do just that.

 public class Sizeof {
   public static void main(String[] args)
    throws Exception {
  // "warm up" all classes/methods that we are going to use:
  runGC();
  usedMemory();

  // array to keep strong references to allocated objects:
  final int count = 10000; // 10000 or so is enough for small ojects
  Object[] objects = new Object[count];

  long heap1 = 0;

  // allocate count+1 objects, discard the first one:
  for (int i = -1; i < count; ++i) {
    Object object;

 //// INSTANTIATE YOUR DATA HERE AND ASSIGN IT TO 'object':


    object=YOUR OBJECT;
 ////end your code here
    if (i >= 0) {
   objects[i] = object;
    }
    else {
   object = null; // discard the "warmup" object
   runGC();
   heap1 = usedMemory(); // take a "before" heap snapshot
    }
  }

  runGC();
  long heap2 = usedMemory(); // take an "after" heap snapshot:

  final int size = Math.round(((float)(heap2 - heap1)) / count);
  System.out.println("'before' heap: " + heap1 +
         ", 'after' heap: " + heap2);
  System.out.println("heap delta: " + (heap2 - heap1) +
         ", {" + objects[0].getClass() + "} size = " + size + " bytes");
   }

   // a helper method for creating Strings of desired length
   // and avoiding getting tricked by String interning:
   public static String createString(final int length) {
  final char[] result = new char[length];
  for (int i = 0; i < length; ++i) {
    result[i] = (char)i;
  }

  return new String(result);
   }

   // this is our way of requesting garbage collection to be run:
   // [how aggressive it is depends on the JVM to a large degree, but
   // it is almost always better than a single Runtime.gc() call]
   private static void runGC()
    throws Exception {
  // for whatever reason it helps to call Runtime.gc()
  // using several method calls:
  for (int r = 0; r < 4; ++r) {
    _runGC();
  }
   }

   private static void _runGC()
    throws Exception {
  long usedMem1 = usedMemory(), usedMem2 = Long.MAX_VALUE;

  for (int i = 0; (usedMem1 < usedMem2) && (i < 1000); ++i) {
    s_runtime.runFinalization();
    s_runtime.gc();
    Thread.currentThread().yield();

    usedMem2 = usedMem1;
    usedMem1 = usedMemory();
  }
   }

   private static long usedMemory() {
  return s_runtime.totalMemory() - s_runtime.freeMemory();
   }

   private static final Runtime s_runtime = Runtime.getRuntime();

 } // end of class
Pete
This is flawed by virtue of the fact that the gc() and runFinalization() methods are not deterministic. I.e.: they are only hints to the runtime to do those actions. It's perfectly legal for a runtime to ignore them.
eqbridges
it is not flawed - they will be ignored sometimes as you say which is why the large number of iterations and averaging of results.
Pete
You could have at least credited the source of this code, even if you've tweaked it a bit: http://www.javaworld.com/javaworld/javatips/jw-javatip130.html
wolfcastle
+2  A: 

Looks like there is already a utility for doing this called Classmexer.

I haven't tried it myself, but I would go that route before I rolled my own.

Robin