Too many open files in one of my java routine.

Question

I have a multithreaded code that has to generated a set of objects and write them to a file. When I run it I sometime get "Too many open files" message in Exception. I have checked the code to make sure that all the file streams are being closed properly. Here is the stack trace.

When I do ulimit -a, open files allowed is set to 1024. We think increasing this number is not a viable option / solution.

 [java] java.io.FileNotFoundException: /export/event_1_0.dtd (Too many open files)
 [java]     at java.io.FileInputStream.open(Native Method)
 [java]     at java.io.FileInputStream.<init>(FileInputStream.java:106)
 [java]     at java.io.FileInputStream.<init>(FileInputStream.java:66)
 [java]     at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
 [java]     at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
 [java]     at java.net.URL.openStream(URL.java:1010)

Now what we have identified so far by looking closely at the list of open files is that the VM is opening same class file multiple times.

/export/BaseEvent.class 236
/export/EventType1BaseEvent.class 60
/export/EventType2BaseEvent.class 48
/export/EventType2.class 30
/export/EventType1.class 14

Where BaseEvent is partent of all the classes and EventType1 ant EventType2 inherits EventType1BaseEvent and EventType2BaseEvent respectively. Why would a class loader load the same class file 200+ times. It seems it is opening up the base class as many time it create any child instance.

Is this normal? Can it be handler any other way apart from increasing the number of open files?

Answer 1

Are you doing anything special with classloaders? If you were doing something interesting such as having a class loader per thread then you might have each loader reading the class file.

Answer 2

The only way I can think of where that would happen is if you created a new class loader instance for each instance of your class.

Are you sure you are not doing something else odd?