utf-8

Django filename from database with non-ascii characters

I'm trying to create a file dynamically in Django: response = HttpResponse(mimetype='text/txt') response['Content-Disposition'] = 'attachment; filename=%s' % filename # UnicodeEncodeError response.write('text') return response If I hardcode the filename it works properly, but if I try to create the filename from DB data that contain...

Japanese text is unrecognizable in phpMyAdmin?

I am phpMyAdmin, why is Japanese text in tables unrecognizable? However, I can output the Japanese text from the tables and it can be displayed correctly. Any idea? How to fix phpMyAdmin? ...

Classic ASP: How to write unicode string data in classic ASP?

How can I show an nvarchar column that stores unicode data (Entered with the zawgyi1 font) in a classic ASP web page? When I retrieve and write the value to the page, it shows "?????". I set my ASP page's content type of UTF-8 with the following meta tag: <META http-equiv="Content-Type" content="text/html; charset=UTF-8"> Unfortunat...

How to convert a NSString to UTF-8 format string in iphone sdk?

I have localization in my app(english,spanish,italian).The client sent me strings files but some characters are strange how do i correct them? i have figured out that the client used Mac OS Roman encoding how do i convert this to utf-8. for example Nürnberg is converted into N√ºrnberg when client send me now i want to reconvert it. ...

Convert UTF-8 octets to unicode code points

Hi, I have a set of UTF-8 octets and I need to convert them back to unicode code points. How can I do this in python. e.g. UTF-8 octet ['0xc5','0x81'] should be converted to 0x141 codepoint. ...

Java: MalformedByteSequenceException (XML)

I'm trying to parse XML using this class. When I type out a simple file, it works fine. <testData> <text> odp </text> </testData> Here is my main public static void main(String[] args) { Xml train = new Xml(args[0], "trainingData"); Xml test = new Xml(args[1], "testData"); } However, when I use the file I got by cop...

PHP UTF-8 charset accentuation

Hi guys, Sorry if this question is already somewhere but I need a more pratical approach: The page is set like this: <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> later in php i use the following line $v .= "<a href='" . $videoEntry->getVideoWatchPageUrl() . "' rel='vidbox' title='" . $videoEntry->getVideoTi...

fix broken german characters

Hello, I have a translation file for German words but I got it broken like this: Wنhlen Sie eine Fachgrِكe is there a tool can fix this in utf-8 mode ? it is for Delphi windows software thanks ...

HTML entities do not render correctly in browser after XSLT transform

I have the following XML: <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <example> <contactInfo> <id>12319221</id> <name>Jerry P</name> <market> <name>Test</name> <phone>800.555.1010</phone> </market> <agent> <name>Test User</name> <emai...

Reading utf-8 characters from a gzip file in python

Hi, I am trying to read a gunzipped file (.gz) in python and am having some trouble. I used the gzip module to read it but the file is encoded as a utf-8 text file so eventually it reads an invalid character and crashes. Does anyone know how to read gzip files encoded as utf-8 files? I know that there's a codecs module that can help ...

Handling UTF-16 in a Django uploaded file

In my Django webapp, in one location users can upload a text file where each line contains a string which will be operated on - the file isn't being stored on the server or anything like that. My code looks like this: roFile = request.FILES['uploadFileName'] ros = roFile.read().strip() ros = ros.split('\n') ros = [t.strip() for t in ro...

Creating files with french characters and encoding.

HI, I am creating a file like so. FileStream temp = File.Create( this.FileName ); Then putting data in the file like so. this.Writer = new StreamWriter( this.Stream ); this.Writer.WriteLine( strMessage ); That code is encapsulated in a class hierarchy but that is the meat and potatoes of it. My problem is this. MSDN says that the ...

Mysql server does not support 4-byte encoded utf8 characters

I got a server error running a Data transfer component from Sql Server to MySql db. Error message reads as follows: [MySql][ODBC 5.1 Driver][mysqld-5.0.67-community-nt-log]Server does not support 4-byte encoded UTF8 characters. Source Sql Server table contain nvarchar columns, target MySql table contain varchar columns. Any expert she...

Converting UTF-8 PostgreSQL DB into WIN-1255 Shapefile

Hi, I have a PostgreSQL\PostGIS spatial database which contains Hebrew text columns. The system runs on Ubuntu, and everything works flawlessly with UTF-8. I am trying to dump some tables into shapefile for a Windows program which can only read Windows-1255 strings. Unfortunately, pgsql2shp has no encoding option, although shp2pgsql ha...

execute a query right after connexion in cakePHP

I want to execute a mysql query right after connecting to the database to enable utf-8 SET NAMES 'utf-8' COLLATE 'utf8_unicode_ci' and i want an answer either for a specific model all for the whole application ...

how can I deal with unicode in PHP without mbstring extension

I am using a shared hosting service to host my site so I can't get direct access to PHP configuration or install any extension. So my problem is with utf-8 strings that can't be processed by standard PHP string functions since I don't have mbstring extension installed on the server. I am looking for another way to deal with unicode strin...

Replacing charachers in a mysql database

I have transfered a big phpbb3 database from a latin1 database to an utf-8 Everything went ok, now I need to change the "strange" characters in the database. Yesterday I found a method that worked, now I can't find it anymore, I tried: update phpbb_topics set topic_title = replace(topic_title, 'Ã', 'à'); update phpbb_topics set topic_ti...

Encoding utf-8 to base64 with accents

Hi, I have some data like this: data1 = ['Agos', '30490349304'] data2 = ['Desir\xc3\xa9','9839483948'] I'm using an API that expects the data encoded in base64, so what I do is: data = data1 string = base64.b64encode("Hi, %s! Your code is %s" % (data[0], data[0])) myXMLRPCCall(string) Which works fine with data1. With data2 the enc...

Ruby 1.9: how to properly upcase/downcase multibyte strings?

So matz took the questionable decision to keep upcase and downcase limited to /[A-Z]/i in ruby 1.9.1. ActiveSupport::Multibyte has long had great i18n case jiggering in ruby 1.8.x via String#mb_chars. However, when tried under ruby 1.9.1, it doesn't seem to work. Here's a simple test script I wrote, along with the output I'm getting: ...

csv utf-8 writer - compability with python2.4

At the bottom of this manual http://docs.python.org/library/csv.html we have example of UnicodeWriter But how can i use this example in python 2.4 I got exception about codecs.getincrementalencoder(encoding)(). Property getincrementalencoder created only in version 2.5. Who can replace this property? Thanks! ...