Deciding on a strategy for paginating Book listings without SQL

Question

I have an ArrayList of Books pulled from different Merchants and sorted in Java, depending on user preferences, according to price or customer reviews:

List<Book> books = new ArrayList<Book>();

This requires me to keep a large chunk of data in memory stored as Java objects at all times. Now that I need to paginate this data into listings that span multiple web pages and allow a user to click on a numbered page link to hop to that segment of the data, what's the best way to do this?

My idea was to have maybe 25 book listings per page and rather than use hyperlinks that submit the form data as a GET request of URL parameters, the page number hyperlinks would simply resubmit the form, passing the requested page number as an additional form POST parameter.

<input type="hidden" id="pageNumber" value="0">
<a href="#" onClick="pageNumber=5; this.form.submit()">Page 5</a>

In that case page 5 would simply be a set of 25 records starting at the 125th (5 * 25) record in the ArrayList and ending at the 149th record in the ArrayList.

Is there a better way to do this?

Answer 1

Refactor your application to let e.g. Hibernate pull out data from the underlying database.

Hibernate can do all the sorting and pagination without you having to keep it all in memory at all times.

Answer 2

IMO the request being a GET or POST shouldn't make much difference, so I'd say do whatever floats your boat (shielding head from RESTful rebuttals). The big thing I'd still be concerned about is keeping that list in memory. Pulling it from separate merchants seems like a good argument for not re-retrieving it each time a page is requested, but personally I'd consider storing those results in a local database anyways, even temporarily. Keeping that much data in memory on your app server will have consequences when you have a lot of concurrent users.

Answer 3

I have a open source library to deal with pagination issue in Java Web application. Here is the link:

http://www.hdpagination.org

It may be an option for you to think about.

If you have any question, feel free to ask.

Answer 4

How many results pages do users generally look at? How big is the data (total or per record)? Is this big list always around (static), or created per-query ?

Instead of returning a page with 25 results, can you output (say) 200 in a JSON array, and use javascript to display n .. n+24 results. If you have all the results on the page, you can also do the sorting there as well. Request a 1x1.gif?user=u1&action=whatever if you want to do user tracking (update ads, etc.) when displaying another page.

Depending on your record size, traffic, user behavior, sending JSON could be more compact than the html generated on the server, so you get

• less bandwidth used
• fewer queries made to the server
• user sees better response because pages update quicker (and server is doing fewer queries)

but you will need to do some analysis to see if this will help you. For example, if more than half of people always look at the second page of results, you might as well ship with that first page results 26-50 as well.

On the other hand, you wouldn't want to send 500 results if no one looks past page 3. Unless you wanted to inflate your traffic numbers. Or you could do something dynamic, like send out smaller pages when there is less bandwidth available. God we live in primitive times.