Escaping non-ASCII characters (or how to remove the BOM?)

Question

I need to create an ANSI text file from an Access recordset that outputs to JSON and YAML. I can write the file, but the output is coming out with the original characters, and I need to escape them. For example, an umlaut-O (ö) should be "\u00f6".

I thought encoding the file as UTF-8 would work, but it doesn't. However, having looked at the file coding again, if you write "UTF-8 without BOM" then everything works.

Does anyone know how to either

a) Write text out as UTF-8 without BOM, or b) Write in ANSI but escaping the non-ASCII characters?

Public Sub testoutput()

Set db = CurrentDb()

str_filename = "anothertest.json"
MyFile = CurrentProject.Path & "\" & str_filename
str_temp = "Hello world here is an ö"

fnum = FreeFile

Open MyFile For Output As fnum
Print #fnum, str_temp
Close #fnum

End Sub

Answer 1

... ok .... i found some example code on how to remove the BOM. I would have thought it would be possible to do this more elegantly when actually writing the text in the first place. Never mind. The following code removes the BOM.

(This was originally posted by Simon Pedersen at http://www.imagemagick.org/discourse-server/viewtopic.php?f=8&t=12705)

' Removes the Byte Order Mark - BOM from a text file with UTF-8 encoding
' The BOM defines that the file was stored with an UTF-8 encoding.

Public Function RemoveBOM(filePath)

    ' Create a reader and a writer
            Dim writer, reader, fileSize
            Set writer = CreateObject("Adodb.Stream")
            Set reader = CreateObject("Adodb.Stream")

    ' Load from the text file we just wrote
            reader.Open
            reader.LoadFromFile filePath

    ' Copy all data from reader to writer, except the BOM
            writer.Mode = 3
            writer.Type = 1
            writer.Open
            reader.Position = 5
            reader.CopyTo writer, -1

    ' Overwrite file
            writer.SaveToFile filePath, 2

    ' Return file name
            RemoveBOM = filePath

    ' Kill objects
            Set writer = Nothing
            Set reader = Nothing
    End Function

It might be useful for someone else.

Answer 2

Thank you for this procedure. Indeed not really nice for you'd expect the stream to have some attribute to disregard writing the BOM.