re: UTF-8, PHP and XML Mysql

Question

This is relating to http://stackoverflow.com/questions/1791082/utf-8-php-and-xml-mysql, which I am still trying to get my head around.

I Have a couple of separate questions that will hopefully help me understand how to resolve the issues I am having.

I am trying to read values from a database and output into a file in UTF-8 format. But I am having encoding issues, so i thought I would strip back all my code ans start with:

$string = "Otivägen";
// then output to a file.

But in vim i cant even enter the that string, every time I paste it in I get Otivägen

I tried to create a blank PHP file with only that string and upload it, but when I cat the file again I get Otivägen.

My questions are ...

1. Why is vim displaying it like this?
2. If the file is downloaded would it display correctly if an application was expecting UTF-8?
3. How can I output this string into a file that will eventually be an XML file in UTF-8 encoding.

My understanding of encoding is limited at the moment, and I am trying to understand it.

Answer 1

There is a lot of confusion associated with encodings in Vim. There are two encoding settings, 'encoding' and 'fileencoding'.

'encoding' is the one that relates to the current vim session - I leave this as 'utf-8' all the time, but then I only use gVim or unicode-enabled terminals.

'fileencoding' is the encoding of the file itself, which is automatically detected or can be overridden with a setting (++enc) or a modeline I believe. It is detected based on the 'fileencodings' option.

Try this:

vim
:set encoding=utf-8
:e ++enc=utf-8 test_file.php
i
$string = "Otiv<Ctrl-K>a:gen";
:w

For more information, see:

:help 'encoding'
:help 'fileencoding'
:help 'fileencodings'
:help ++enc
:help modeline

See also http://vim.wikia.com/wiki/Category%3AEncoding

Answer 2

1. Vim supports UTF-8 from version 6.0. Your system is likely not using UTF-8 by default - you're likely seeing UTF-8 text displayed in ASCII (or another 8-bit fixed encoding).

2. It should. Set the encoding on the file to UTF-8 when you serve it.

3. Any file writing function would accept this - UTF-8 is just a sequence of bytes.

Answer 3

This might not be a vim issue - if your terminal software is not set to utf-8 then you'll see the above issues regardless of what vim is doing.