Why am I getting this simplejson exception?

Question

Why does Django give me this exception

[(7, u'Acura'), (18, u'Alfa Romeo'), ...] is not JSON serializable

When I try

data = VehicleMake.objects.filter(model__start_year__gte=request.GET.get('year',0)).values_list('id','name')
return HttpResponse(simplejson.dumps(data, ensure_ascii=False), mimetype='application/json')

?

It's just a simple list of tuples. It works with my other hard-coded list that's in almost exactly the same format. Is it because the strings are unicode? How do I handle that?

---

It works fine when I encode it as a dict:

def get_makes(request):
    year = request.GET.get('year',0)
    data = VehicleMake.objects.filter(model__start_year__lte=year, model__stop_year__gte=year).order_by('name').distinct().values_list('id','name')
    return HttpResponse(simplejson.dumps(odict(data), ensure_ascii=False), mimetype='application/json')

Some makes have accented characters... could that be it? Yes, the list is big (~900 makes total).

Answer 1

This seems to work fine:

In [28]: a = [(7, u'Acura'), (18, u'Alfa Romeo'),]

In [29]: simplejson.dumps(a, ensure_ascii=False)
Out[29]: u'[[7, "Acura"], [18, "Alfa Romeo"]]'

So it's not the first couple of tuples. You'll need to dig deeper in the records list to narrow down the issue. If it's large, perhaps take some slices of the data list and try encoding those, to see if the error occurs in any particular segment.

UPDATE: OK, it's probably because your data object is a QuerySet and simplejson doesn't handle that. Try using django's serialize instead. (Or coerce the data to a list.)

from django.core import serializers
json_serializer = serializers.get_serializer("json")()
json_serializer.serialize(data, ensure_ascii=False, stream=response)

Answer 2

Ticket #6234 claims that leaving out ensure_ascii=False will resolve the problem (but I am not sure if the problem is really understood):

Simply omitting ensure_ascii parameter resolves the issue even though it makes no sense.

Answer 3

Instead of

return HttpResponse(simplejson.dumps(data, ensure_ascii=False), mimetype='application/json')

Use list(data) and modify your Javascript to work with it.

for(i in values) {
    $select.append('<option value="'+values[i][0]+'">'+values[i][1]+'</option>');
}