Read Unicode files C++

Question

Hi all,

I have a simple question to ask. I have a UTF 16 text file to read wich starts with FFFE. What are the C++ tools to deal with this kind of file? I just want to read it, filter some lines, and display the result.

It looks simple, but I just have experience in work with plain ascci files and I'm in the hurry. I'm using VS C++, but I'm not want to work with managed C++.

Regards

Here a put a very simple example

wifstream file; 
file.open("C:\\appLog.txt", ios::in);

wchar_t buffer[2048]; 
file.seekg(2);
file.getline(buffer, bSize-1);

wprintf(L"%s\n", buffer);
file.close();

Answer 1

The FFFE is just the initial BOM (byte order mark). Just read from the file like you normally do, but into a wide char buffer.

Answer 2

For what it's worth, I think I've read you have to use a Microsoft function which allows you to specfiy the encoding.

http://msdn.microsoft.com/en-us/library/z5hh6ee9(VS.80).aspx

Answer 3

You can use fgetws, which reads 16-bit characters. Your file is in little-endian,byte order. Since x86 machines are also little-endian you should be able to handle the file without much trouble. When you want to do output, use fwprintf.

Also, I agree more information could be useful. For instance, you may be using a library that abstracts away some of this.

Answer 4

Since you are in the hurry, use ifstream in binary mode and do your job. I had the same problems with you and this saved my day. (it is not a recommended solution, of course, its just a hack)

  ifstream file; 
  file.open("k:/test.txt", ifstream::in|ifstream::binary);

  wchar_t buffer[2048]; 
  file.seekg(2);
  file.read((char*)buffer, line_length);
  wprintf(L"%s\n", buffer);
  file.close();

Answer 5

Upgrading an STL-based application to use Unicode