Django <-> SQL Server 2005, text encoding problem

Question

Hello,

I'm trying to store Django data on MS SQL Server 2005 using:

http://code.google.com/p/django-pyodbc/ (pyodbc + FreeTDS)

As long as I'm storing string consist of ASCII characters everything is ok. When I'm using unicode (ex. '\xc5\x82'), django throws ProgrammingError on:

ProgrammingError at /admin/cli/punktrejestracji/add/
('42000', '[42000] [FreeTDS][SQL Server]The incoming tabular data stream (TDS) protocol stream is incorrect. The stream ended unexpectedly. (4002) (SQLExecDirectW)')

last element of trace is:

params  ('\xc5\x82',)
self    <django.db.backends.sql_server.pyodbc.base.CursorWrapper object at 0x92ef8ec>
sql 'SELECT (1) AS [a] FROM [cli_punktrejestracji] WHERE [cli_punktrejestracji].[adres] = ? '

BTW http://code.google.com/p/django-mssql/ doesn't seems to work under Linux, django-mssql needs pythoncom library. Am I right?

Answer 1

We use Django with SQL Server 2005. We found the same problem you did.

What ODBC driver are you using? FreeTDS?

We tried finding a good ODBC driver for linux/unix to use that would not throw the error above (and others) when unicode would come into play - and failed miserably. None of the drivers we tested - at least three, I can dig the names up if you would like - had any success in dealing with unicode strings via django-pyodbc.

What we ended up doing, sad as it might sound, was to decide to run Django on a Windows server (Apache + mod_wsgi) and use the Microsoft's SQL Native ODBC driver.

It works just fine - unicode wise - when we do that.

Answer 2

OK, the solution was found. In file freetds.conf there is

client charset = UTF-8

and it works exactly like it should.