High performance chart for lots of data in javascript

Question

I need a high performance charting component. The underlying technology used is not specific; could be javascript, flash or java for what it's worth.

I need to solve these issues:

1. I need to decide on the right kind of chart for this kind of data:

001. testA 002,003
002. testB 003,004
003. testC 004
004. testD

Currently, I'm think of a network kind of view:

http://www.connectedaction.net/wp-content/uploads/2009/09/2009-September-NodeXL-CHI-2010-Tag-Network.png

http://visualgadgets.blogspot.com/2008/06/graphs-and-networks.html

2. The chart needs to excel on performance:

To date, I have over 604 million records, and I expect it to grow to a billion in the next few days. That's a lot to visualize.

3. The chart needs to be accessible from javascript

Considering issue 2, I will need to visualize the chart at one go (during application sartup) and then update it gradually. By "update" I mean "add records"; records (and links between nodes) will not be removed.

---

Don't know what else I could add, that's pretty much all.

---

Edit: I got some useful advice; I could use preprocessed data to show the full chart and then only show realtime details when one zooms in accordingly.

However, given my above data format, it's a bit difficult to do this considering there might be circular references. I'll have to think this through.

Meanwhile, I still need advice on a good charting component wfor network/topology diagrams.

Answer 1

This leaves out serverside charting since that will take up a lot of resources on the server if you maintain a high traffic website. The logical thing would be to let the client side do the chart processing.

Unless you display a lot of charts on a single page, performance should not be that big of a deal client side. However, to use the data, you will need to reformat it for ANY charting toolkit/framework to read.

Raphael is a good visualisation toolkit, there are probably toolkits that extend Raphael to do more specific charting. Afaik, you need to do a lot of hand coding yourself when using Raphael out of the box (though still only once for all your billion charts ;oP).

Here's a link to a top 5 jQuery chart list.

More can be found with a short google session ;)

Answer 2

another option to think about is a preprocessed database. i did it a few years ago on sql server, we set up a second DB server with some very complicated DTS packages and sql scripts that took the massive amount of data we were working with and summarized it to let the crystal reports server be able to handle serving the reports.

you loose the ability to have real time reports, but if thats not a requirement it worked well for us and you reduce the load on whatever report server you are using and reduces the load on your production database for running reports and lets it concentrate on running the application it also reduces the load on your client browsers because you don't have to send your chart this huge chunk of data.

as far as visualizing, i've heard a lot of good things about JavaFX and its charting capabilities.

Answer 3

With regards to charting, I found the following candidates:

ProtoVis and jsViz

Both are javascript based, hence quite slow on rendering (I guess?). I think I'll try ProtoVis first.