views:

69

answers:

1

I'm hitting what appears (to me) strange behavior when I pull data from the google datastore over JDO. In particular, the query executes quickly (say 100 ms), but finding the size of the resulting List<> takes about one second! Indeed, whatever operation I try to perform on the resulting list takes about a second. Has anybody seen this behavior? Is it expected? Unusual? Any way around it?

PersistenceManager pm = PMF.getPersistenceManager();
Query q = pm.newQuery("select from " + Person.class.getName());
System.out.println("getting all at " + System.currentTimeMillis());
mcs = (List<Person>) q.execute();
System.out.println("got all at " + System.currentTimeMillis());
int size = mcs.size();
System.out.println("size was " + size + " at " + System.currentTimeMillis());
getting all at 1271549139441
got all at 1271549139578
size was 850 at 1271549141071

-B

+1  A: 

Calling execute() executes the query, but doesn't return all the results - results are fetched as-needed when you access the list. Calling .size() requires the datastore to fetch and count all the results, which is a very inefficient operation!

Nick Johnson
Thanks Nick. Can you point me to a resource that will explain this behavior in more detail? In particular, I'm curious about what sparks round-trip communication to the database server, and what's performed locally.
Bosh
Also, is the implication really that it takes a full 1.6 SECONDS to fetch 850 rows in a very small table?
Bosh
Unfortunately, I'm not aware of any current documentation that goes into detail about how JDO or JPA make database calls. As far as runtime goes, that depends highly on your model - how large it is and how many fields there are - as deserialization time could easily exceed the database roundtrip time. Also, the time taken to execute a query doesn't depend on the size of your table, only on the size of the result set. You should generally structure your app so it only fetches results that the user will see.
Nick Johnson
My model's pretty simple, with just a Long key, three enumeration fields, a double, and two strings.Sadly, I'm displaying a list of all entities for the user to see :-(
Bosh
Why not paginate the list? Few users can handle a page with nearly 1000 results anyway!
Nick Johnson
Well, the UI helps the user visualize data (e.g. histograms with bin size defined on the client side).
Bosh