InputB vs. Get; code pages; slow reading on unix server

Question

Hi,

We have been using the usual code to read in a complete file into a string to then parse in VB6. The files are ANSI text but encoded using whatever code page the user was in at the time (we have Chinese and English users for example). This is the code

Open FileName For Binary As nFileUnit
sContents = StrConv(InputB(LOF(nFileUnit), nFileUnit), vbUnicode)

However, we have discovered this is VERY slow reading a file from a server running unix/linux, particularly when the ownership of the file is not the same as the process doing the reading.

I have rewritten the above using Get and discovered it is much faster and does not suffer from any issues with file ownership. I appreciate that this might be solved by reconfiguring the server somehow, but I think since deiscovering even without that issue, the Get method is still much faster than InputB I'd like to replace my existing code using Get.

I wonder if someone could tell me if this will really do the same thing. In particular, is it correctly doing the ANSI to Unicode conversion and will this always be true. My testing suggests the following replacement code does the same thing but faster:

  Open FileName For Binary As nFileUnit
  sContents = String(LOF(nFileUnit), " ")
  Get #nFileUnit, , sContents

I also realise I could use a byte array, but again my tests suggest the above is simpler and works. So how does the buffer work correctly (if you believe the online help for Get it talks of characters returned - clearly this would cause problems when reading in an ANSI file written on the Chinese code page with 2-byte Chinese characters in it).

The following might be of interest becuase the InputB approach is commonly given as the method to read a complete file, but it is much slower, examples

Reading 380Kb file across the network from the unix server

InputB (file owned) = 0.875 sec

InputB (not owned) = 72.8 sec

Get (either) = 0.0156 sec

Reading a 9Mb file across the network from the unix server

InputB (file owned) = 19.65 sec

Get (either) = 0.42 sec

Thanks Jonathan

Answer 1

InputB() is CVar(InputB$()), and is known to be horribly slow. My suspicion is that InputB$() reads the bytes and converts them to Unicode using the current codepage via some stock logic for reading text from disk, then does another conversion back to ANSI using the current codepage.

You might be far ahead to use ADODB.Stream.LoadFromFile() to load complete ANSI text files. You can set the .Type = adTypeText and .Charset = the appropriate ANSI encoding as required to read Unicode back out of it via .ReadText(x) where x can be a number of bytes, or adReadAll or adReadLine. For line reading you can set .LineSeparator to adCR, adCRLF, or adLF as required.

Many Charset values are supported: KOI8 for Cyrillic, Big5 for Chinese, etc.