Loaded data truncated when using nsIFileInputStream & nsIConverterInputStream

Question

I'm working on a project (BrowserIO - go to browserio dot googlecode dot com if you want to check out the code and work on it. Help welcome!) in which I'm using Firefox's nsIFileInputStream in tandem with nsIConverterInputStream, per their example (https://developer.mozilla.org/en/Code%5Fsnippets/File%5FI%2F%2FO#Simple), but only a portion of the full data is being loaded. The code is:

var file = Components.classes["@mozilla.org/file/local;1"].createInstance(Components.interfaces.nsILocalFile);
file.initWithPath(path);
var data = "";

var fstream = Components.classes["@mozilla.org/network/file-input-stream;1"].createInstance(Components.interfaces.nsIFileInputStream);
var cstream = Components.classes["@mozilla.org/intl/converter-input-stream;1"].createInstance(Components.interfaces.nsIConverterInputStream);

fstream.init(file, -1, 0, 0);
cstream.init(fstream, "UTF-8", 0, 0); // you can use another encoding here if you wish

var str = {};
cstream.readString(-1, str); // read the whole file and put it in str.value
data = str.value;

cstream.close(); // this closes fstream

If you want to see this behavior, checkout the code from the BrowserIO project page, and use Firebug to set a breakpoint at the data = str.value; line in file_io.js. Then select a text file from the list, and click the "Open" button. In Firebug, in the watch panel set a watch for str.value. Look at the file... It should be truncated, unless it's really short.

For reference, the code above is the main body of the openFile() function in trunk/scripts/file_io.js.

Anybody have any clue what's happening with this?

Answer 1

See nsIConverterInputStream; basically, -1 doesn't mean "give me everything" but rather "give me the default amount", which the docs claim is 8192.

More generally, if you want to exhaust the contents of an input stream, you have to loop until it's empty. Nothing in any of the stream contracts guarantees that the amount of data returned by a call is the entirety of the contents of the stream; it could even return less than it has immediately available if it wanted.

Answer 2

I discovered how to do the file read without converting, to avoid issues from not knowing the file encoding type. The answer is to use nsIScriptableInputStream with nsIFileInputStream:

var sstream = Components.classes["@mozilla.org/scriptableinputstream;1"].createInstance(Components.interfaces.nsIScriptableInputStream);
fstream.init(file, 0x01, 0004, 0);
sstream.init(fstream);
data = sstream.read(sstream.available());