Why does Django give me different results for the same query?

Question

For a mock web service I wrote a little Django app, that serves as a web API, which my android application queries. When I make requests tp the API, I am also able to hand over an offset and limit to only have the really necessary data transmitted. Anyway, I ran into the problem, that Django gives me different results for the same query to the API. It seems as if the results are returned round robin.

This is the Django code that will be run:

def getMetaForCategory(request, offset, limit):
    if request.method == "GET":
        result = { "meta_information": [] }

        categoryIDs = request.GET.getlist("category_ids[]")

        categorySet = set(toInt(categoryIDs))
        categories = Category.objects.filter(id__in = categoryIDs)

        metaSet = set([])

        for category in categories:
            metaSet = metaSet | set(category.meta_information.all())

        metaList = list(metaSet)
        metaList.sort()

        for meta in metaList[int(offset):int(limit)]: 
            relatedCategoryIDs = getIDs(meta.category_set.all())

            item = {
                "_id": meta.id,
                "name": meta.name,
                "type": meta.type,
                "categories": list(categorySet & set(relatedCategoryIDs))
            }

            result['meta_information'].append(item)

        return HttpResponse(content = simplejson.dumps(result), mimetype = "application/json")
    else:
        return HttpResponse(status = 403)

What happens is the following: If all MetaInformation objects would be Foo, Bar, Baz and Blib and I would set the limit to 0:2, then I would get [Foo, Bar] with the first request and with the exact same request the method would return [Baz, Blib] when I run it for the second time.

Does anyone see what I am doing wrong here? Or is it the Django cache that somehow gets into my way?

Answer 1

I think the difficulty is that you are using a set to store your objects, and slicing that - and sets have no ordering (they are like dictionaries in that way). So, the results from your query are in fact indeterminate.

There are various implementations of ordered sets around - you could look into using one of them. However, I must say that I think you are doing a lot of unnecessary and expensive unique-ifying and sorting in Python, when most of this could be done directly by the database. For instance, you seem to be trying to get the unique list of Metas that are related to the categories you pass. Well, this could be done in a single ORM query:

meta_list = MetaInformation.objects.filter(category__id__in=categoryIDs)

and you could then drop the set, looping and sorting commands.