How to free virtual memory ?

Question

I have a crawler application (with C#) that downloads pages from web . The application take more virtual memory , even i dispose every object and even use GC.Collect() .

This , have 10 thread and each thread has a socket that downloads pages . In each thread , i have a byte[] buffer that store content of page , and have an string str_content that i store in it , content of page in string . I have a Sytem.Timer that every 3 second chech , if each thread has been stopped , assign it new thread and start it.

I use dispose method and even use GC.Collect() in my application , but in 3 hour my application take 500 MB on virtual memory (500 MB on private bytes in Process explorer) . Then my system will be hang and i should restart my pc .

1. would it be rude , If i assign my byte[] and string to null ?

2. Is there any way that i use to free virtual memory ?

Thanks .

Answer 1

First of all, you SHOULDN'T be calling gc.collect() anyway, as it is a costly call, and shouldn't be necessary.

If you are seeing growth AND you are still calling gc.collect() you have resources that still have references, thus they can't be collected.

I would start looking at your code and making sure that all of your objects are declared at the proper scope, that you are using the Using Statement syntax to ensure that items implementing IDisposable are properly cleaned up and do a full review of your code.

The next step would be to take a tool like ANTS profiler or the like and look at what is actually stored in memory.

Answer 2

It would take exploration of your code to see where your memory leaks are. It's likely in events that you're tying into (either manually or automatically) that cause apparently-out-of-scope objects to not get properly disposed.

Answer 3

In C# (and in Java) as long as you have a reference to the object, the program environment assumes you are still using the object. Calls to free memory will only free unused objects. The key is to stop using the object.

Odds are excellent you have something like:

Object mine = new Object();

the key is that you also need something like:

mine = null;

to signal that the "mine" object is no longer being used. Typically these problems don't happen in code blocks like this because once you leave the block, the variables aren't accessible anymore:

public void process() {
  Object mine = new Object();
}

Typically these problems happen in code blocks like this, because the Collection accumulates objects over time:

static List tasks = new ArrayList();

public void process(String item) {
  tasks.add(item);
}

The key is that without a corresponding tasks.remove(item) the list will hold references to the items forever thwarting garbage collection efforts.