tags:

views:

177

answers:

3

Hi all,

I'm having a problem with some Java NIO code running on an iSeries box (JDK 1.5). Basically the code is splitting a file up into chunks part of a file to another smaller files. The same code has been operating on other iSeries boxes for some time with no problems. Here's the code snippet:

//copy original data file content to temp file
long startPos = dataFile.length() - remaining;
long transferSize = maxSizeBytes - size;
size += inChannel.transferTo(startPos, transferSize, outChannel); //exception here
remaining -= size;

Here's the stack trace:

Caused by: java.io.IOException: Operation not supported. Map failed
 at java.lang.Throwable.<init>(Throwable.java:196)
 at java.lang.Exception.<init>(Exception.java:41)
 at java.io.IOException.<init>(IOException.java:40)
 at sun.nio.ch.FileChannelImpl.map0(Native Method)
 at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:742)
at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:448)
at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:521)

... 11 more

I've done some investigation and the causes so far (file permissions of parent directory, out of memory, shared memory control QSHRMEMCTL switched off, use of SAN) have all proved unsuccessful.

Anyone have any experience of this particular problem?

Thanks, Brad.

A: 

According to this thread (which supports the argument with reference to the NIO source), the likely cause is an out of memory condition.

Jonathan Feinberg
Yeah have been looking at that. I'm assuming that source is the Sun JDK implementation so not necessarily the same? Assuming it is though, we managed to provoke the same error using a test class running against a small file (couple K) and it still failed. another factor, the initial error was encountered with 2gig heap size and data file of 376 bytes! Still possible I guess, but seems unlikely. Maybe I should try some -Xmx experimentation.
Brad
The out of memory condition would likely be on the part of the OS, not the Java heap (since you're trying to do mmap).
Jonathan Feinberg
+1  A: 

It sticks in my mind that reaching the file handle limit can result in non-obvious exceptions being raised by the JVM.

Check to see if you have enough file handles available. ulimit will tell you how many are at your disposal. (Of course, you'll want to know this number for the user the JVM is running under if it is a daemon.) This problem would also be system/user specific, which kinda fits your description of the fact this runs elsewhere just fine.

Stu Thompson
Hi Stu, interesting point, I'll check this.
Brad
Stu - checked this value under the target user account, the returned value was "unlimited".
Brad
That's not it then! Bummer.
Stu Thompson
A: 

OK, another shot: Could you post more of your code? Like the initialization of all viables and your control loop. Even though you are saying works elsewhere, I have to wonder about a few things and so have disregarded that.

1: long startPos = dataFile.length() - remaining;
2: long transferSize = maxSizeBytes - size;
3: size += inChannel.transferTo(startPos, transferSize, outChannel); //exception here
4: remaining -= size;
  1. Is remaining initialized to dataFile.length() outside of your loop? If not, then that is going to blow up from the start all the time.
  2. size might be probably better named bytesTransfered. I found myself getting a bit mixed up with that.
  3. You don't need both remaining and size variables. One or the other should suffice
  4. Is maxSizeBytes initialized to a value <= dataFile.length()
  5. Can you log startPos, transferSize, and dataFile.length()? I'm wondering if maybe you are inadvertently
    1. passing a transferSize larger than dataFile.length()
    2. passing a startPos larger than dataFile.length()
Stu Thompson
Stu, thanks for looking at this again. I did some tweaking of the code re that transfer size mentioned in the first answer but it didn't help. We've got round it by adding in a non-NIO code path which is selectable on systems where we see this problems. Thanks for your help and for taking an interest.
Brad