R - idiomatic way to deal with lists of data frames

Question

I have 30 runs of data, each stored in a separate CSV file, runi.csv, i = 0:29.

Let's say I want to collect them all into a list. Best way I know how to do this is

runs = list()
for (i in 1:30) { runs[[i]] = read.csv(paste("run", i-1, ".csv")); }

Now let's further say that each of these data frames stored in the list has the same column layouts and that I'm interested in the column identified by "x" and the column identified by "y".

What is the easiest way to plot all 30 runs' worth of (x, y) pairs? Here's how I would currently do it (and I feel there must be a better way):

xList = list()
yList = list()
for (i in 1:30) { xList[[i]] = runs[[i]]$x; yList[[i]] = runs[[i]]$y; }
matplot(x=as.data.frame(xList), y=as.data.frame(yList))

This gets even more painful when I'm trying to do transformations to the data; I can't figure out how to apply a function to a specific column of each data frame stored in a list.

Any help here would be extremely helpful.

Answer 1

It is probably best to use an l*ply function (from plyr) or lapply when dealing with lists like this.

The easiest way to do the import is probably like so:

library(plyr)
runs <- llply(paste("run",1:30,".csv",sep=""), read.csv)

Here's one way to plot them:

# some dummy data
runs <- list(a=data.frame(x=1:5, y=rnorm(5)), b=data.frame(x=1:5, y=rnorm(5)))
par(mfrow=c((length(runs)/2),2));
l_ply(1:length(runs), function(i) { plot(runs[[i]]$x, runs[[i]]$y) })

Of course, you can also output this to another device (e.g. pdf) and not use par().

Answer 2

You would probably be much better off creating one data frame with all the data. For example, add the run number when importing (runs[[i]] = data.frame(read.csv(paste("run", i-1, ".csv")), Run=i)), and then do alldata <- do.call(rbind, runs).

Now you can use lattice or ggplot2 to make plots. For example to get a scatterplot of all runs using different colors by run do:

library(ggplot2)
qplot(x, y, colour=Run, data=alldata, geom="point")