FileSystemObject - Reading Unicode Files

Question

Classic ASP, VBScript context.

A lot of articles including this Microsoft one, say you cannot use FileSystemObject to read Unicode files.

I encountered this issue a while back, so switched to using ADODB.Stream instead, per the ReadText example here, instead of using FileSystemObject.OpenTextFile (which does accept a final parameter indicating whether to open the file as unicode, but actually doesn't work).

However, ADODB.Stream results in a world of pain when trying to read a file on a UNC fileshare (permissions-related issue). So, investigating this, I stumbled across the following approach which works a) with unicode files, and b) across UNC fileshares:

dim fso, file, stream
set fso = Server.CreateObject("Scripting.FileSystemObject")
set file = fso.GetFile("\\SomeServer\Somefile.txt")
set stream = file.OpenAsTextStream(ForReading,-1) '-1 = unicode

This is using the FSO to read a unicode file without any apparent problem, so I'm confused as to all the references, including MS, saying you can't use the FSO to read unicode files.

Has anyone else used this approach for reading unicode files? Are there any hidden gotchas I'm missing, or can you really actually read unicode files using FSO?

Answer 1

I'd say if it works, use it ;-)

I notice the MS article you refer to is from the Windows 2000 (!) scripting guide. Maybe it's obsolete.

Answer 2

Yes that documentation is out of date. The scripting component did go through a set of changes in its early days (some of them were breaking changes if you were using early binding) however since at least WK2000 SP4 and XP SP2 it has been very stable.

Just be careful what you mean by unicode. Sometimes the word unicode is used more broadly and can cover any encoding of unicode. FSO does not read for example UTF8 encodings of unicode. For that you would need to fall back on ADODB.Stream.

Answer 3

I think MS does not officially state that it supports unicode because:

1. It does not detect unicode files using the byte-order mark at the start of the file, and
2. It only supports Little-Endian UTF-16 unicode files (and you need to remove the byte order mark if present).

Here is some sample code that I have been using successfully (for a few years) to auto-detect and read unicode files with FSO (assuming they are little-endian and contain the BOM):

'Detect Unicode Files
Set Stream = FSO.OpenTextFile(ScriptFolderObject.Path & "\" & FileName, 1, False)
intAsc1Chr = Asc(Stream.Read(1))
intAsc2Chr = Asc(Stream.Read(1))
Stream.Close
If intAsc1Chr = 255 And intAsc2Chr = 254 Then 
    OpenAsUnicode = True
Else
    OpenAsUnicode = False
End If

'Get script content
Set Stream = FSO.OpenTextFile(ScriptFolderObject.Path & "\" & FileName, 1, 0, OpenAsUnicode)
TextContent = Stream.ReadAll()
Stream.Close