I am interested in knowing if there is any alternative to rrdtool for logging time series data. I am looking at something that can scale for a large number of devices to monitor.
From what I read on this subject, rrdtool is I/O bound when you hit it with large amounts of data. Since I envision this to scale to a very large number of devices to monitor, I am curious if there's any alternative that would not choke on I/O. Preferable SQL based, but not necessarily.
Thanks
A friend of mine did some work a while ago on a SQL backend to store round robin data: http://rrs.decibel.org
However, I suspect that since you're asking about "devices to monitor", you may be looking for a more complete solution.
If I/O performance is the main worry then you want to look into something like rrdcached which is available in the current version (1.4) of the RRDTools.
The I/O overhead is not a function of the data being written, after all each value 8 bytes per data source. The I/O bandwidth comes from the fact a whole sector (typically 4k) needs to be read in before being written out. Suddenly to write 8 bytes you have read/written 8k bytes.
The rrdcached coalesces all these write together so when an RRD is updated the ratio of useful data (actual DS values) to wasted data (the spare bytes in the sector) is reduced.
All the RRDTools will automatically work with rrdcached when they detect it running (via an environment variable). This allows them to trigger flushes when needed, for example when generating a graph from the data.
While switching to an SQL based solution may help consider the extra I/O that will be required to support SQL. Considering you don't tend to use RRD data in that sort of random access pattern a database is a bit of a sledgehammer for the problem. While sticking with RRDTool will keep access to all the eco-system of tools that understand and can work with the files, which is useful especially if you are already familiar with it.