Is it conceivable that the rate at which new objects are created by a Java program somewhat exceeds the rate at which these objects are garbage collected in the young generation ?
...Thus causing an ever greater backlog of new objects sitting on the heap until a full gc is triggered ?
Any metrics I could collect to identify/verify this scenario ?
The objects created have a maximum lifespan of around 1 second. Most are alive for about 400 milliseconds, on average.
Thanks.
It is conceivable ...
I don't think the HotSpot serial, parallel or CMS collectors would behave this way. You can read all about them here, here and here (for the G1 collector).
There is one situation where objects get promoted without a young generation GC cycle. That is when the collector determines that the old space is too full to hold any more promoted objects. Then the serial or parallel collector will perform a full collection of the young and old generations.
Any metrics I could collect to identify/verify this scenario ?
The GC logs might reveal this. I don't know.
When the GC kicks in and moves objects out from Young (to S0 or S1) then dead object are not copied and therefore thrown away. The objects are moved a number of times back and forth between S0 and S1 until they die or they become old enough to be transfered to Old space.
S0 and S1 is storage space between the Young(Eden) and the Old space.
The GC runs so long there are objects to copy, throwing away all dead ones. Full GC is executed when memory actually needs to be recovered, example objects needs to be removed from Old space.
The fastest way to clean Young space is actually to wait until most of the objects are dead, cause then you do not have to copy as many to S0|S1 and later to Old.
So filling up the memory, and then taking the penalty of a longer GC run is actually more effective then letting the GC run with very short intervals. This is why you see your memory growing and growing until it is almost full, and then it all drops down to almost nothing again.
A full GC is StopTheWorld this is needed to rearrange the objects on the heap in Old space, it is not needed in Young space. If young space becomes to full to fast objects can be promoted to Old space. This is not a good case and this is the case you are thinking of.
To avoid Full GCs, you can calculate how much memory you will need in Young Space with the creation rate you have to make sure that objects are not promoted before necessary. It is also possible to tune the size of the S0 and S1 space and how many times objects will bounce between them before they end up in Old Space (requiring a Full GC to be removed).
A good link but a bit old is this one on Tuning Garbage Collection