I have a java application that uses a lot of memory when used, but when the program is not being used, the memory usage doesnt go down.
Is there a way to force Java to release this memory? Because this memory is not needed at that time, I can understand to reserve a small amount of memory, but Java just reserves all the memory it ever uses. It also reuses this memory later but there must be a way to force Java to release it when its not needed.
System.gc is not working.
System.gc has no guarantees about if it should free any memory when ran. See http://stackoverflow.com/questions/2414105/why-is-it-a-bad-practice-to-call-system-gc
Try tweaking the Xmx JVM arg down if it is set to a large value and take a look in JConsole to see what's going on with memory usage and GC activity. Normally you'd see a saw tooth pattern.
You might also want to use a profiler to see where the memory is being used and to identify any leaks.
One of two things is happening:
1) Your application is leaking references. Are you sure that you aren't hanging on to objects when you'll no longer need them? If you do, Java must maintain them in memory.
2) Java's working just fine. You get no benefit from memory that you aren't using.
What memory do you mean? If it is RAM (as opposed to the amount of used heap space of the Java VM itself) then this might be normal. It is a relatively expensive operation to allocate memory so once the JVM got some it is quite reluctant to give it back even if it is not needed at the time.
Have you considered using a memory profiler? If you don't have access to one, you can start with capturing a bunch of jmap -histo <pid> and writing a script to figure the differences.