Java, IO - fastest way to remove file.

Question

Hey guys,

My problem is that I have an app which is writing a lot of relatively (100-500kb) small CSV files (tens and hundreds of thousands ). Content of those files then get loaded in database via sql loader call (its oracle db) and this is what I have to live with.

So, I need to remove those small files time to time to prevent them from eating up all space. I would like to attach that to the activity which writes those files and loads them into db as a last finalize step.

My Question is -- how in java can one remove a bunch of small files with less overhead on performance?

Thanks in advance! Michael

Answer 1

Well, file.delete() should suffice (it is internally implemented as a native method)

Answer 2

You may find it an order of magnitude faster if you shell out and have the system delete them. You'd have to be able to hit a stopping point (where no files were being processed) then shell out and delete "*" or . or whatever it is for your OS.

(Note, this makes your program VERY os dependent!)

Be sure on Windows and Mac that you are bypassing the trashcan feature!

The nice thing about del . or rm * is that they SHOULD batch the operation rather than repeatedly opening, modifying and closing the directory.

You might also write filenames with a pattern like a001, a002, a003, ... and when you reach a999 you go to b001 and delete a*.

Answer 3

I'd suggest checking the Apache Commons IO library. They have some pretty helpful methods for deleting files in the FileUtils class.