utf-8

Counterintuitive results from setting the connection character set

I have nearly completed the task of overhauling my web app to be properly "UTF-8 aware". I have found, though, that if I set the connection character set to utf8 using mysqli_set_charset, the result is that output appears incorrectly (indeed it appears as though the page's character encoding had been misidentified), whereas if I do not s...

Mac version of Python doesn't support UTF-8 in curses module? [Solved]

Hi, I'm trying to display a lot of unicode text in my curses application. My development machine is MacOSx 10.6 and I use the default python shipped with Apple. Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin When I added unicode text to the screen, the screen all messed up. I tried to ...

Problems displaying French accented characters in UTF-8

I'm working on a French language site built in CakePHP. I have tried multiple functions to try and convert the text into UTF-8 and display properly, but have had no success so far - any accented letters are displaying as a black diamond with a question mark. They do display correctly when I change the char set in the browser to ISO-885...

Java/XSLT character encoding

I am using XSLT to display some &#160; (space) and &#163; (£) signs but they all come out as question marks (unrecognised characters). My encoding is set to UTF-8 in the HTML and the XSLT header looks like this <xsl:stylesheet version="1.0" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns="http://www.w3.org/1999/xhtml" xmlns:xs...

Problem in converting string to puny code (in PHP, using phlyLabs's punycode string converter)

hi, i'm using the code from here: http://phlymail.com/en/downloads/idna/download/ and built a function like this (from the example): function convert_to_punycode($inputstring) { $IDN = new idna_convert(); // The input string, if input is not UTF-8 or UCS-4, it must be converted before $inputstringutf8 = utf8_encode($inputst...

Converting from utf-16 to utf-8 in Python 3

I'm programming in Python 3 and I'm having a small problem which I can't find any reference to it on the net. As far as I understand the default string in is utf-16, but I must work with utf-8, I can't find the command that will convert from the default one to utf-8. I'd appreciate your help very much. ...

Error with two byte UTF-8 character in UPDATE statement for MySQL database

An update-statement seems to work only with one or three byte long UTF-8 characters. My test code def sql = sql.newInstance('jdbc:mysql://.../...?useUnicode=true&characterEncoding=UTF-8', '...', '...', 'com.mysql.jdbc.Driver') String value = 'β' sql.execute('UPDATE Kldb_SynonymVersion SET synonyms=? WHERE id=11940', [value]...

unicode preg_replace problem in php

I've got the string $result = "bei einer Temperatur, die etwa 20 bis 60°C unterhalb des Schmelzpunktes der kristallinen Modifikation" which comes straight from a MySQL table. The table, and the php headers are both set to UTF-8 I want to strip the 'degree' symbol: http://en.wikipedia.org/wiki/Degree_symbol and replace it with the wor...

php problem with encoding in a cli script reading id3 tags

i am trying to get a php cli script to go through a folder, get the id3 tags, which are in utf8 in cyrillic and but it in the database. when i execute the script i get the raw utf in the DB fields like "&#192;&#235;&#225;&#229;&#237;&#224;" here is the script <? set_time_limit(0); include('classes/adodb5/adodb.inc.php'); include ('cla...

Google App Engine DataStore Text UTF-8 Encoding Problem

I'm building a gwt app that stores the text of random webpages in a datastore text field. Often the text is formatted UTF-8. All the files of my app are stored as UTF-8 and when I run the application on my local machine the entire process works fine. UTF-8 text is stored as such and retrievable ftom the local version of the app engine ...

Character encoding problem from Facebook JSON to HTML via PHP

I'm getting a JSON encoded array from Facebook which contains: [{"message":"D\u011bkujeme Zuzana Boh\u00e1\u010dov\u00e1 za na\u0161i novou profilovou fotku :-)\nWe thank Zuzana Boh\u00e1\u010dov\u00e1 for our new profile picture :-)"}] When I decode the JSON and output the contents I get: DÄ›kujeme Zuzana BoháÄová za...

UnicodeString to char* (UTF-8)

I am using the ICU library in C++ on OS X. All of my strings are UnicodeStrings, but I need to use system calls like fopen, fread and so forth. These functions take const char* or char* as arguments. I have read that OS X supports UTF-8 internally, so that all I need to do is convert my UnicodeString to UTF-8, but I don't know how to do ...

How to prevent non-ascii characters from being saved in Drupal database?

I've noticed that in Drupal after some users enter content into the body of a node, characters like quotes and apostrophes get saved as: ’ “ This is due the user entering odd characters or something... Does this mean the Drupal database hasn't been configured for utf8? How can this be corrected so ’ is actually saved as just '? ...

Submitted character encoding -- _charset_ hidden field

For our web app, we have multiple HTML pages containing text areas. All of our pages are rendered with an ISO-8859-1 charset. When the page is accessed through IE6 on a Windows machine and special characters such as a "smart quote" are copied in to the text area, some of our pages submit the page using the Windows 1252 character encodi...

Can Codeigniter write_file write a UTF-8 file?

Hello, In CodeIgniter, I'm writing an XML file using write_file function. write_file('data.xml', $str_xml); But the encoding of the file is always ANSI. How can I have this file be encoded to UTF-8 automatically? ...

Json - Mysql Charset Problem

I convert my php array to json using json_encode, Than I record this json data to my database .While json_encode encodes my data , It also converts strings to utf8 , this changes special characters to something like '/u011f' or '/u0131' . But when I write this converted data to mysql database , Backslashes are disappearing in field, so ...

PHP include html page charset problem

Hello, after querying a mysql db using the code below i have generated an html file: $myFile = "page.htm"; $fh = fopen($myFile, 'w') or die("can't open file"); fwrite($fh, $row['text']); fclose($fh); On the msql db the text is encoded using utf8_general_ci. But i need to include it in a php web page as shown below: <?includ...

UTF-8 encoding on filehandle in Perl

I'm applying UTF-8 encoding to STDIN and STDOUT. However how do I make sure that I apply UTF-8 encoding to the file that I pass to my code below (<> will read from a file instead of STDIN if a text file is passed on the command line) in as few lines as possible. use open qw(:std :utf8) while (<>) { print; } ...

PHP image_jpeg dont' work if file is UTF-8, works if it's ANSI

Hi. I use this class to crop, resize my image: http://www.phpclasses.org/package/4268-PHP-Resize-crop-rotate-flip-flop-and-grey-images.html It uses GD. The problem is that I for example do this: $img = new ImageTransform; $img->view('resize', 'foo.jpg', '500x400', true); // true argument is $cache = true Now I can just point <img src=...

Why declare unicode by string in python?

I'm still learning python and I have a doubt: In python 2.6.x I usually declare encoding in the file header like this (as in PEP 0263) # -*- coding: utf-8 -*- After that, my strings are written as usual: a = "A normal string without declared Unicode" But everytime I see a python project code, the encoding is not declared at the he...