I have a web application that functions as a dashboard, allowing a user to see summaries of historical data to view trends, etc. As an extension to this, I want to allow the user to drill-down into the historical data if they so wish.
What this will amount to is a table of time-value pairs, showing the time a particular data point was recorded as well as its associated value. The issue is that there could be a very large amount of data on record (millions of points is entirely possible), which means it wouldn't work to load all the data up front and display it to the user.
So far, my best idea is to implement the table of time-value pairs with "infinite scrolling" - i.e. the first x points are loaded and then as the user scrolls down the next x points are fetched and so on until the user reaches the end of the data (with a fixed-size cache of points, so that I don't run out of memory). The issue with this approach, though, is that there's no way to get to the end of the data without first fetching every point in between.
Another option would be to have a navigation interface with forward and back buttons, as well as some sort of date chooser so the user can quickly navigate to any data point they wish. This solves the problem of the first idea, but this navigation interface is far more clunky and less intuitive.
My question is: is there some happy medium? A UI where it's clean and easy to navigate through the data but at the same time provide some method of jumping to a specific point in time?
Edits based on comments below:
The development environment is flex and the database is SQL Anywhere, though the back-end logistics are less important to me than a clean, functional UI.
I agree that paging through historical data is not the most helpful thing. Unfortunately, though, some of the historical data doesn't lend itself to summaries. For example, there is string-based data where at time t1 the value is "foo" but at t2 the value is "bar" - I don't have any good ideas for how to summarize data like that.