Django annotate to show aggregate sums and counts pre-slicing of the QuerySet

Question

I have a fairly complex QuerySet which users a good deal of annotated values to get some counts and sums across the entire record set. The resulting rows are grouped and for each group I show the sum of this column, or the count of that column etc.

The problem is that when I slice the query set in order to paginate the data, the query is evaluate and the counts and sums now relate to the sliced query set instead of the full one.

Is there a way to avoid this, i.e., keep the sums/counts showing across the entire rows returned yet still returning a sliced query set?

I tried to see if I can cause the query set to execute first and only slice it after it had executed, but couldn't find how to do that so I don't know if that would work (would it?)

Many thanks, Harel

Answer 1

You should be able to get the queryset to evaluate by doing:

qs = list(qs)

At that point it's no longer a queryset, just a regular Python list, which can be sliced as usual.

The problem with doing this for pagination, of course, is that no matter what page is viewed you are returning the entire set of records from the database. That could be a real problem if your dataset is large. Given your requirements it sounds like that may be unavoidable; your query either deals with all the records, or with just some of them. Without seeing more details it's hard to know for sure.

Answer 2

I ended up running a separate query to get my sums, and return its result along with the actual page in question. The amount of data I'm dealing with doesn't really allow me to work on entire query sets like that. Thanks Carl!