windbg "Free" object type

Question

Monitoring my program's Virtual Bytes usage while it is running showed that by doing some kind operations, the virtual bytes usage goes up by about 1GB in about 5 minutes. The program deals with tcp sockets and high data transfer throughput between them (~800Mbps).

Loading a dump file of the program in windbg showed that the reason for the very high and fast memory usage is about 1GB of "free" objects. Indeed, when I call the garbage collector (gen 0, 1 & 2) from the console screen of the program (after getting it to this state) it frees up about 1GB of memory usage.

I'm trying to understand what exactly are these free objects and why aren't they garbage collected by the garbage collector automatically.

Edit: One suggestion was that I may be creating the objects in the Large Object Heap and it becomes fragmanted but this is not the case as I've seen that all of the "free" objects sits in Gen 2 Heap.

Other suggestion was that maybe Gen 2 Heap gets fragmented because of pinned objects but if this was the case, GC.Collect wouldn't fix the problem but it actually does so I believe this is not the case as well.

What I suspect from the discussion with Paul is that the memory does gets freed but is from some reason returned to the OS rarely or only when I manually call GC.Collect.

Answer 1

They are not free 'objects', they are free space. .NET does not release memory it has used back to the operating system immediately. Any free blocks can be used for subsequent object allocations, providing they fit inside the free block (otherwise the heap has to be extended by asking the operating system to allocate more memory).

The garbage collector makes efforts to combine free space into large usable blocks by compacting generation 2. This is not always possible: for example an application may pin objects which will potentially prevent the garbage collector from combining free space by moving the live objects to the front of the heap. If this happens a lot, the application's memory will be broken up into useless small blocks and this affect is known as 'heap fragmentation'.

In addition, there is a Large Object Heap (LOH) in which larger objects are allocated. The rationale is that there is a cost associated with heap compaction as data must be copied around and so the LOH is not compacted, avoiding these costs. However, the flipside is that the LOH can become easily fragmented, with small, less useful blocks of free memory interspersed between live objects.

I would suggest running a dumpheap -stat. This command will report at the end of the list any fragmented blocks. You can then dump those blocks to get an idea of what is happening.

Answer 2

By the way, it looks like you have a well-known problem (at least among socket gurus) that most socket servers get in .Net. Paul has already touched on what it means. To elaborate more, what is going wrong is that during a Read/Write on the socket the buffer gets pinned - which means the GC isn't allowed to move it around (as such your heap fragments). Coversant (who pioneered the solution) were seeing an OutOfMemoryException when their actual memory usage was only about 500MB (due to such heavy fragmentation). Fixing it is another story entirely.

What you want to do is at application start up allocate a very large amount of buffers (I am currently doing 50MB). You will find the new ArraySegment<T> (v2.0) and ConcurrentQueue<T> (v4.0) classes particularly useful when writing this. There are a few lock-free queues floating around on the tubes if you are not using v4.0 yet.

// Pseudo-code.
ArraySegment<byte> CheckOut()
{
  ArraySegment<byte> result;
  while(!_queue.TryDequeue(out result))
    GrowBufferQueue(); //Enqueue a bunch more buffers.
  return result;
}

void CheckOut(ArraySegment<byte> buffer)
{
  _queue.Enqueue(buffer);
}

void GrowBufferQueue()
{
  // Verify this, I did throw it together in 30s.
  // Allocates nearly 2MB. You might want to tweak that.
  for(var j = 0; j < 5; j++)
  {
    var buffer = new byte[409600]; // 4096 = page size on Windows.
    for(var i = 0; i < 409600; i += 4096)
      _queue.Enqueue(new ArraySegment<byte>(buffer, i, 4096));
  }
}

Following this you will need to subclass NetworkStream and swap out the incoming buffer with one from your buffer pool. Buffer::BlockCopy will help performance (don't use Array::Copy). This is complicated and hairy; especially if you make it async-capable.

If you are not layering streams (e.g. SSLStream <--> DeflateStream <--> XmlWriter etc.) you should use the new socket async pattern in .Net 4.0; which has more efficiency around the IAsyncResults that get passed around. Because you are not layering streams you have full control over the buffers that get used - so you don't need to go the NetworkStream subclass route.