How can I get this File Writing code to work with Unicode (Delphi)

Question

I had some code before I moved to Unicode and Delphi 2009 that appended some text to a log file a line at a time:

procedure AppendToLogFile(S: string);
// this function adds our log line to our shared log file
// Doing it this way allows Wordpad to open it at the same time.
var F, C1 : dword;
begin
  if LogFileName <> '' then begin
    F := CreateFileA(Pchar(LogFileName), GENERIC_READ or GENERIC_WRITE, 0, nil, OPEN_ALWAYS, 0, 0);
    if F <> 0 then begin
      SetFilePointer(F, 0, nil, FILE_END);
      S := S + #13#10;
      WriteFile(F, Pchar(S)^, Length(S), C1, nil);
      CloseHandle(F);
    end;
  end;
end;

But CreateFileA and WriteFile are binary file handlers and are not appropriate for Unicode.

I need to get something to do the equivalent under Delphi 2009 and be able to handle Unicode.

The reason why I'm opening and writing and then closing the file for each line is simply so that other programs (such as WordPad) can open the file and read it while the log is being written.

I have been experimenting with TFileStream and TextWriter but there is very little documentation on them and few examples.

Specifically, I'm not sure if they're appropriate for this constant opening and closing of the file. Also I'm not sure if they can make the file available for reading while they have it opened for writing.

Does anyone know of a how I can do this in Delphi 2009 or later?

---

Conclusion:

Ryan's answer was the simplest and the one that led me to my solution. With his solution, you also have to write the BOM and convert the string to UTF8 (as in my comment to his answer) and then that worked just fine.

But then I went one step further and investigated TStreamWriter. That is the equivalent of the .NET function of the same name. It understands Unicode and provides very clean code.

My final code is:

procedure AppendToLogFile(S: string);
// this function adds our log line to our shared log file
// Doing it this way allows Wordpad to open it at the same time.
var F: TStreamWriter;
begin
  if LogFileName <> '' then begin
    F := TStreamWriter.Create(LogFileName, true, TEncoding.UTF8);
    try
      F.WriteLine(S);
    finally
      F.Free;
  end;
end;

Finally, the other aspect I discovered is if you are appending a lot of lines (e.g. 1000 or more), then the appending to the file takes longer and longer and it becomes quite inefficient.

So I ended up not recreating and freeing the LogFile each time. Instead I keep it open and then it is very fast. The only thing I can't seem to do is allow viewing of the file with notepad while it is being created.

Answer 1

Char and string are Wide since D2009. Thus you should use CreateFile instead of CreateFileA!

If you werite the string you shoudl use Length( s ) * sizeof( Char ) as the byte length and not only Length( s ). because of the widechar issue. If you want to write ansi chars, you should define s as AnsiString or UTF8String and use sizeof( AnsiChar ) as a multiplier.

Why are you using the Windows API function instead of TFileStream defined in classes.pas?

Answer 2

Try this little function I whipped up just for you.

procedure AppendToLog(filename,line:String);
var
  fs:TFileStream;
  ansiline:AnsiString;
  amode:Integer;
begin
  if not FileExists(filename) then
      amode := fmCreate
  else
      amode := fmOpenReadWrite;
fs := TFileStream.Create(filename,{mode}amode);
try
if (amode<>fmCreate) then
   fs.Seek(fs.Size,0); {go to the end, append}

 ansiline := AnsiString(line)+AnsiChar(#13)+AnsiChar(#10);
 fs.WriteBuffer(PAnsiChar(ansiline)^,Length(ansiline));
finally
   fs.Free;
end;

Also, try this UTF8 version:

procedure AppendToLogUTF8(filename, line: UnicodeString);
var
    fs: TFileStream;
    preamble:TBytes;
    outpututf8: RawByteString;
    amode: Integer;
  begin
    if not FileExists(filename) then
      amode := fmCreate
    else
      amode := fmOpenReadWrite;
    fs := TFileStream.Create(filename, { mode } amode, fmShareDenyWrite);
    { sharing mode allows read during our writes }
    try

      {internal Char (UTF16) codepoint, to UTF8 encoding conversion:}
      outpututf8 := Utf8Encode(line); // this converts UnicodeString to WideString, sadly.

      if (amode = fmCreate) then
      begin
          preamble := TEncoding.UTF8.GetPreamble;
          fs.WriteBuffer( PAnsiChar(preamble)^, Length(preamble));
      end
      else
      begin
        fs.Seek(fs.Size, 0); { go to the end, append }
      end;

      outpututf8 := outpututf8 + AnsiChar(#13) + AnsiChar(#10);
      fs.WriteBuffer(PAnsiChar(outpututf8)^, Length(outpututf8));
    finally
      fs.Free;
    end;
end;

Answer 3

For logging purposes why use Streams at all?

Why not use TextFiles? Here is a very simple example of one of my logging routines.

procedure LogToFile(Data:string);
var
  wLogFile: TextFile;
begin
  AssignFile(wLogFile, 'C:\MyTextFile.Log');
  {$I-}
  if FileExists('C:\MyTextFile.Log') then
    Append(wLogFile)
  else     
    ReWrite(wLogFile); 
  WriteLn(wLogfile, S);
  CloseFile(wLogFile);
  {$I+}
  IOResult; //Used to clear any possible remaining I/O errors 
end;

I actually have a fairly extensive logging unit that uses critical sections for thread safety, can optionally be used for internal logging via the OutputDebugString command as well as logging specified sections of code through the use of sectional identifiers.

If anyone is interested I'll gladly share the code unit here.