UnicodeEncodeError: 'ascii' codec can't encode character

Question

When uploading files with non-ASCII characters I get UnicodeEncodeError:

Exception Type: UnicodeEncodeError at /admin/studio/newsitem/add/
Exception Value: 'ascii' codec can't encode character u'\xf8' in position 78: ordinal not in range(128)

See full stack trace.

I run Django 1.2 with MySQL and nginx and FastCGI.

This is a problem that is fixed according to the Django Trac database, but I still have the problem. Any suggestions on how to fix are welcome.

EDIT: This is my image field:

image = models.ImageField(_('image'), upload_to='uploads/images', max_length=100)

Answer 1

It's hard to say without seeing a little more code but it looks to be related to this question: http://stackoverflow.com/questions/2457087/.

Looking through the Django ticket mentioned it would seem you should follow something similar to the deployment docs on "If you get a UnicodeEncodeError":
http://docs.djangoproject.com/en/dev/howto/deployment/modpython/#if-you-get-a-unicodeencodeerror

(I know this is for Apache/mod_python but my guess would be it's the same root issue of file system encoding not being UTF-8 and there is a similar fix when using nginx)

EDIT: From what I can tell this nginx module would be the equivalent fix: http://wiki.nginx.org/NginxHttpCharsetModule

Answer 2

In situations where you must display a unicode string in a place that only accepts ascii (like the console or as a path) you must tell Python that you want it to replace the non ascii characters best effort.

>> problem_str = u'This is not all ascii\xf8 man'
>> safe_str = problem_str.encode('ascii', 'ignore')
>> safe_str
'This is not all ascii man'

Encoding issues are prevented in the admin by the cautious handing of Django templating, but if you have ever added custom columns and forgotten to convert the values to ascii, or you override the str method for a model and forget to do this, you will get the same error, preventing template rendering.

If this string were saved into your (hopefully utf8) database there would be no problem, it looks like you are trying to upload a file that uses the title of an entity that has a non ascii character.