Java NIO framework for filesystems instead of networks?

Question

There are several high quality frameworks that hide the complexity of NIO based network programming (mina, netty, grizzly, etc.). Are there similar frameworks that simplify NIO based file-system programming?

For example, as a learning exercise, I would like to implement a disk backed Map based on this (awesome!) article: http://www.javaworld.com/javaworld/jw-01-1999/jw-01-step.html.

Answer 1

No (but...)

But that is because Java's NIO FileChannel and MappedByteBuffer are not nearly as complex or difficult to understand and use as the networking and Selector stuff java.nio.

Here is an example of creating a disk backed map (known as a 'mapped byte buffer' in NIO-land) that would be appropriate for your exercise:

File file = new File("/Users/stu/mybigfile.bin");
FileChannel fc = (new FileInputStream(file)).getChannel(); 
MappedByteBuffer buf = fc.map(MapMode.READ_WRITE, 0, file.length());

You can access the buffer like any other Buffer. Data moves magically and quickly between disk and memory, all managed by Java and the underlying OS's virtual memory management system. You do have a degree of control of this, though. E.g.: MappedByteBuffer's .force() ('Forces any changes made to this buffer's content to be written to the storage device containing the mapped file.') and .load() ('Loads this buffer's content into physical memory.') I've never needed these personally.

Answer 2

To add to @Stu's comment. It worth noting that socket connections do not have all their data at once but instead may need to support many slow connections (esp connections which are open but no data is sent yet)

However for files, all the data is available at once and you typically only need to open a few files at a time to get maximum performance (often one at a time is fine) If you are loading data from multiple drives (rare) or from multiple servers (very rare) or multiple network interfaces (even rarer) you might fine accessing a few files at a time improves performance. Even then the complexity is not high and you can just create a thread for each file you are loading.

The only occasion where files are complicated is reading log files. This complicated as the file can grow in size as you read it. You can reach the end of the files and later find more data. Also log files can be rotated meaning the file you had open is no longer the file you want. Even so this is not very difficult to deal with and a fairly rare requirement.