Question mark characters displaying within text, why is this?

Answer 1

Unicode or other character set characters falling through?

I have seen similar "strange" characters show up on sites I have worked on often when copy is copied from an email or some other docuement format (e.g. word) into a text editor. The editor can display the non ascii characters but the browser can't. For the website, I would suggest looking up the html entity code for the character and inserting that instead ... or switch to more standard ones.

Answer 2

Your browser hasn't interpretted the encoding of the page correctly (either because you've forced it to a particular setting, or the page is set incorrectly), and thus cannot display some of the characters.

Answer 3

This is going to be something to do with character encodings.

Are you sure the mirrored site has the same properties with regards to character encodings as your main server?

Depending on what sort of server you have, this may be a property of the server process itself, or it could be an environment variable.

For example, if this is a UNIX environment, perhaps try comparing LANG or LC_ALL?

See also here

Answer 4

The following articles will be useful

http://dev.mysql.com/doc/refman/5.0/en/charset-syntax.html

http://dev.mysql.com/doc/refman/5.0/en/charset-connection.html

After you connect to the database issue the following command:

SET NAMES 'utf8';

Ensure that your web page also uses the UTF-8 encoding:

PHP also offers several function that will be useful for conversions:

http://us3.php.net/manual/en/function.iconv.php

http://us.php.net/mb_convert_encoding

Answer 5

Check the character set being emitted by your mirrored server. There appears to be a difference from that to the main server -- the live site appears to be outputting Unicode, where the mirror is not. Also, it's usually a good idea to scrub Unicode characters in your incoming content and replace them with their appropriate HTML entities.

Your specific issue regards "smart quotes," "em dashes" and "en dashes." I know you can replace em dashes with — and n-dashes with – (which should be done on the input side of your database); I don't know what the correct replacement for the smart quotes would be. (I usually just replace all curly single quotes with ' and all curly double quotes with " ... Typography geeks may feel free to shoot me on sight.)

I should note that some browsers are more forgiving than others with this issue -- Internet Explorer on Windows tends to auto-magically detect and "fix" this; Firefox and most other browsers display the question marks.

Answer 6

I usually curse MS word and then run the following Wscript.

// replace with path to a file that needs cleaning
PATH = "test.html"

var go=WScript.CreateObject("Scripting.FileSystemObject");
var content=go.GetFile(PATH).OpenAsTextStream().ReadAll();
var out=go.CreateTextFile("clean-"+PATH, true);

// symbols
content=content.replace(/“/g,'"');
content=content.replace(/”/g,'"');
content=content.replace(/’/g,"'");
content=content.replace(/–/g,"-");
content=content.replace(/©/g,"©");
content=content.replace(/®/g,"®");
content=content.replace(/°/g,"°");
content=content.replace(/¶/g,"<p>");
content=content.replace(/¿/g,"&iquest;");
content=content.replace(/¡/g,'&iexcl;');
content=content.replace(/¢/g,'&cent;');
content=content.replace(/£/g,'&pound;');
content=content.replace(/¥/g,'&yen;');

out.Write(content);