International Fonts Display Issue with UTF-8

Question

Hi

We have developed a PHP-MySQL application in two languages - English and Gujarati. The Gujarati language contains symbols that need unicode UTF-8 encoding for proper display.

The application runs perfectly on my windows based localhost and on my Linux based testing server on the web.

But when I transfer the application to the client's webserver (linux redhat based), the Gujarati characters are not displayed properly.

I have checked the raw data from both the databases (on my webserver and on the client's webserver) - it is exactly the same. However the display is different. On my server the fonts are displayed perfectly, but when I try to access the client's copy of the app, the display of Guajarati font is all wrong.

Since I am testing both these installation instances from the same machine and the same browser, the issue is not of browser incompatability or the code. I believe that there is some server setting that needs to be done, which I am missing out.

Can you help please.

Thanks

Vinayak

UPDATE

Thanks. We have tried the apache and php settings suggestions given by the SO community members. But the problem remains.

To breakdown the issue we have looked at the different stages that the data is passing through.

The two databases (at the client's end and at at our end) are identical. There is no difference in them.

The next step in this app is that a query is run which recovers the data, creates an xml file and saves it.

This XML file is then displayed using a PHP page.

We have identified that the problem is that there is a difference in the XML file being created. The code used for creating the XML file is as below:

function fnCreateTestXML($testid)
{
 $objQuery = new clsQuery();
 $objTest = new clsTest();
 $setnames = $objQuery->fnSelectRecords("tbl_testsets", "setnumber", "");
 $queryresultstests = $objQuery->fnSelectRecords("tbl_tests", "", "");
 if($queryresultstests)
 {

  foreach($queryresultstests as $queryresulttest)
  {
   foreach($setnames as $setname)
   {
    //Creating Root node test and set its attribute 
    $doc = new DomDocument('1.0','utf-8');
    $root = $doc->createElement('test');
    $root = $doc->appendChild($root);
    //and so on

//Saving XML on the disk
    $xml_create = $doc->saveXML();
    $testname = "testsxml.xml";
    $xml_string = $doc->save($testname);

Any ideas??

Thanks

Vinayak

Answer 1

Make sure, that the response headers are correct - the Content-type should have UTF-8 in it.

Answer 2

Check the character set settings on the DB instance on the client's machine:

SHOW VARIABLES LIKE 'character_set%';
SHOW VARIABLES LIKE 'collation%';

Before executing any text fetching queries, try executing:

SET NAMES utf8
SET CHARACTER SET utf8

If that works, you might want to add the following to the my.cnf on the client's machine:

[mysqld]
collation_server=utf8_unicode_ci
character_set_server=utf8
default-character-set=utf8
default-collation=utf8_general_ci
collation-server=utf8_general_ci

Answer 3

Since you've stated that it is working in your development environments and not on your clients, you might want to check the clients Apache's AddDefaultCharset and set this to UTF-8, if it's not already. (Assuming that they're using Apache.)

I tend to make sure the following steps are checked

1. PHP Header is sent in UTF-8
2. Meta tag is set to UTF-8 (Content-Type)
3. Storage is set to UTF-8
4. Server output is set to UTF-8
5. Make sure your php code files are encoded in UTF8 with BOM (Byte Order Mark)

Answer 4

The answer almost certainly lies in the headers being sent with the web pages. To diagnose issues like this, I've found it useful to install the firefox addon "Live HTTP Headers".

Install that addon, then turn it on and reload a page from the client's webserver, and from your own.

What you'll probably see is that the page served by your webserver has the header:

Content-Type: text/html; charset=UTF-8

Whereas when served by the client webserver it says:

Content-Type: text/html

The way I would recommend fixing this is for you to ensure that you explicitly set the header to specify utf-8 in every page of your application. This then insulates your application from future configuration changes on the client's end.

To do this, call

header('Content-type: text/html; charset=utf-8');

on each page before sending any data.