information seemingly coming out of mysqldb incorrectly, python django

Question

In a latin-1 database i have '\222\222\223\225', when I try to pull this field from the django models I get back u'\u2019\u2019\u201c\u2022'.

from django.db import connection
(Pdb)
cursor = connection.cursor()
(Pdb)
cursor.execute("SELECT Password from campaignusers WHERE UserID=26")
(Pdb)
row = cursor.fetchone()

So I step into that and get into

/usr/local/python2.5/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-linux-i686.egg/MySQLdb/cursors.py(327)fetchone()->(u'\u2019...1c\u2022',)

I can't step further into this because its an egg but it seems that the MySQL python driver is interpreting the data not as latin-1.

Anyone have any clue whats going on?

Answer 1

A little browsing of already-asked questions would have led you to UTF-8 latin-1 conversion issues, which was asked and answered yesterday.

BTW, I couldn't remember the exact title, so I just googled on django+'\222\222\223\225' and found it. Remember, kids, Google Is Your Friend (tm).

Answer 2

Django uses UTF-8, unless you define DEFAULT_CHARSET being something other. Be aware that defining other charset will require you to encode all your templates in this charset and this charset will pop from here to there, like email encoding, in sitemaps and feeds and so on. So, IMO, the best you can do is to go UTF-8, this will save you much headaches with Django (internally it's all unicode, the problems are on the borders of your app, like templates and input).