utf-8

How To Store Hmong Characters In MySQL Database

I've read a number of articles on storing multi-language strings in MySQL, but I can't seem to find anything specific (or credible) on Hmong. I have no trouble with latin (European) languages, but if someone could enlighten me on Hmong, that would be terrific. Thanks! P.S. Using PHP for the scripting, if anyone cares. ...

Unicode problem in NetBeans 6.7.1

When I type any text on Georgian language. NetBeans shows it like question marks. I'm using Windows7(georgian keyboard). I've also tried in Eclipse, but there is no such problem (everything works fine). Then I've tried to open my Eclipse project folder in NetBeans with some html files and the question marks appeared again. (The html fi...

iText encoding problem

I have encoding problem with iText (http://www.lowagie.com/iText/). I load data from database and insert it as html to pdf with iText, for some reason my non-english (Finnish ä,ö etc) characters don't show up correctly. Following example shows how insert text to html: text = "<p>" + data + "</p>"; HTMLWorker htmlWorker = new HTMLWorke...

Convert from hex string to unicode

How can i convert the 'dead' string to an unicode string u'\xde\xad'? Doing this: from binascii import unhexlify out = ''.join(x for x in [unhexlify('de'), unhexlify('ad')]) creates a <type 'str'> string '\xde\xad' Trying to use the Unicode.join() like this: from binascii import unhexlify out = ''.join(x for x in [u'', unhexlify('d...

Retrieving and displaying UTF-8 from a .CSV in Python

Basically I have been having real fun with this today. I have this data file called test.csv which is encoded as UTF-8: "Nguyễn", 0.500 "Trần", 0.250 "Lê", 0.250 Now I am attempting to read it with this code and it displays all funny like this: Trần Now I have gone through all the Python docs for 2.6 which is the one I use and I can...

Why does ContentResult controller in ASP.NET MVC return UTF-16 when UTF-8 specified?

I have an ActionResult that returns XML for an embedded device. The relevant code is: return Content(someString, "text/xml", Encoding.UTF8); Even though UTF-8 is specified, the resulting XML is: <?xml version="1.0" encoding="utf-16"?> The ASP.NET MVC is compiled as AnyCPU and runs on a Windows 2008 server. Why is it not returni...

Character with encoding UTF8 has no equivalent in WIN1252

I am getting the following exception: Caused by: org.postgresql.util.PSQLException: ERROR: character 0xefbfbd of encoding "UTF8" has no equivalent in "WIN1252" Is there a way to eradicate such characters, either via SQL or programmatically? (SQL solution should be preferred). I was thinking of connecting to the DB using WIN1252 but i...

Is "SET CHARACTER SET utf8" necessary?

Hi all! I´m rewritting our database class (PDO based), and got stuck at this. I´ve been taught to both use SET NAMES utf8 and SET CHARACTER SET utf8 when working with UTF-8 in PHP and MySQL. In PDO I now want to use the PDO::MYSQL_ATTR_INIT_COMMAND parameter, but it only supports one query. Is SET CHARACTER SET utf8 necessary? ...

utf-8 to iso-8859-1 encoding problem

I'm trying preview the latest post from an rss feed on another website. The feed is UTF-8 encoded, whilst the website is ISO-8859-1 encoded. When displaying the title, I'm using; $post_title = 'Blogging – does it pay the bills?'; echo mb_convert_encoding($post_title, 'iso-8859-1','utf-8'); // returns: Blogging ? does it pay the b...

Mysql's LIKE is missbehaving with Hebrew and backslashes, why?

I have the following SQL query which returns the correct results: SELECT * FROM `tags` WHERE tag_name = 'בית\\\"ר-ירושלים' If I change it to SELECT * FROM `tags` WHERE tag_name LIKE 'בית\\\"ר-ירושלים' or to SELECT * FROM `tags` WHERE tag_name LIKE 'בית\\\"ר-ירושלים%' It doesn't work. It will work if I remove all the backslashes ...

PHP Japanese Strings getting set to ????????????

I have a PHP file with one simple echo function: echo 'アクセスは撥ねりません。'; but when I access that page i get this: ???????????? Can someone help me? I also have my page encoding set to UTF-8, and I know it, because all of the browsers i used said so. I also do this before the echo function: mb_internal_encoding('UTF-8'); What does th...

C# and utf8_decode

Is there a C# utf8_decode equivalent? ...

struts2 request encoding

I am sending a XML in HTTP POST body. Question: Does struts2 support processing request in utf-8 encoding? Reference: http://www.experts-exchange.com/Programming/Languages/Java/Q%5F24061148.html (Around bottom of the page) ...

Change Website Character encoding from iso-8859-1 to UTF-8

About 2 years ago I made the mistake of starting a large website using iso-8859-1. I now am having issues with some characters, especially when sending data to the server using ajax. Because of this, I would like to switch to using UTF-8. What issues do you see coming from this? I know I would have to search the site to look for ch...

Delphi 7 Personal, MySQL using libmysql.dll + UTF8

Hi, I'm using Delphi 7 Personal. To access MySQL database I'm using libmysql.dll + very simple wrapper, which is good enough for me. Except one thing ... it doesn't seem to handle Utf8... is that possible somehow to pass Utf8 strings from libmysql to Delphi? Please keep in mind I'm not using commercial delphi, this means no ADO / dbExpr...

Converting a database from one character encoding to another

I have a MYSQL database. Text is currently stored in charset latin1, collation latin1_swedish_ci. These are the defaults and it wasn't a problem back in the day when the database was originally created. I want to switch over to UTF8 so the text encoding in the database matches out text encoding used elsewhere on the web site that uses t...

Ruby: How to convert a string to binary and write it to file

The data is a UTF-8 string: data = 'BZh91AY&SY\x94$|\x0e\x00\x00\x00\x81\x00\x03$ \x00!\x9ah3M\x13<]\xc9\x14\xe1BBP\x91\xf08' I have tried File.open("data.bz2", "wb").write(data.unpack('a*')) with all kinds of variations for unpack put have had no success. I just get the string in the file not the UTF-8 encoded binary data in the stri...

How to best configure PHP to handle a UTF-8 website

What extensions would you recommend and how should php be best configured to create a website that uses utf-8 encoding for everything. eg... Page output is utf-8 forms submit data encoded in utf-8 internal processing of string data (eg when talking to a database) are all in utf-8 as well. It seems that php does not really cope well w...

&nbsp; not being displayed properly

I am having problem in displaying the &nbsp; in my web page, after using utf8_decode() in PHP it gets displayed as �. Please help urgent. i have been using <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"&gt; <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en...

How can I convert an input file to UTF-8 encoding in Perl?

I already know how to convert the non-utf8-encoded content of a file line by line to UTF-8 encode, using something like the following code: # outfile.txt is in GB-2312 encode open my $filter,"<",'c:/outfile.txt'; while(<$filter>){ #convert each line of outfile.txt to UTF-8 encoding $_ = Encode::decode("gb2312", $_); ...} ...