Django : json serialize a queryset which uses defer() or only()

Question

Now I've been using json serializer and it works great.

I had to modify my queries where I started using the only() & defer() filters, like so -

retObj = OBJModel.objects.defer("create_dt").filter(loged_in_dt__gte=dtStart)

After I've done the above, suddenly the json serializer is returning empty fields -

{"pk": 19047, "model": "OBJModel_deferred_create_dt", "fields": {}}

If I remove the defer(), the serializer gives all the data correctly.

import json
from django.utils import simplejson
from django.core import serializers
json_serializer = serializers.get_serializer("json")()
retObj = OBJModel.objects.defer("create_dt").filter(loged_in_dt__gte=dtStart)
json_serializer.serialize(retObj, ensure_ascii=False)

I've scratched my head on this for a while now. Any insight would be great.

NOTE : I am using django 1.1

Answer 1

I'm confused as to how you would expect the serializer to behave in regards to deferred fields... It is possible that I'm missing something...

The doc for json serialization says:

Be aware that if you're serializing using that module directly, not all Django output can be passed unmodified to simplejson. In particular, lazy translation objects need a special encoder written for them.

The doc is talking about lazy translations, but I think any lazy operation applies.

I think what you are seeing is just the correct output if you haven't written a special encoder of some sort that will take care of obtaining (accessing) the proper value for the deferred fields.

Edit after your comment: Ah, I was missing the fact that no other fields are being encoded. What are the types of the other fields? Could we see your model? FK and M2M fields are handled differently by the default encoder - but I don't see anything on django.core.serializers.python.Serializer or django.core.serializers.json.Serializer that would explain why the other non-deferred fields are not encoding...

Edit after some further investigation: The OBJModel_deferred_create_dt in your json payload above made me dig a bit further. It seems that is the result of a call to django.db.models.queryutils.deferred_class_factory() from the __reduce__ method in the django's base Model class. The deferred_class_factory():

Returns a class object that is a copy of "model" with the specified "attrs" being replaced with DeferredAttribute objects. The "pk_value" ties the deferred attributes to a particular instance of the model.

This is where things get murky (to me!): In reality the pickling is being done against a proxy model for your actual OBJModel. This proxy model should return - when asked - the non-deferred fields of the original model. But in your case it appears it isn't.

I'll try to set up a small test and see if I can replicate the problem.