Reading 3-dimensional datasets into R

Question

Gnuplot allows for three dimensional datasets, which are a set of tables separated by empty lines, for instance:

54.32,16.17,7.42,4.28,3.09,2.11,1.66,1.22,0.99,0.82,7.9

54.63,15.50,8.53,5.31,3.75,1.66,1.14,0.83,0.94,0.52,7.18
56.49,16.67,6.38,3.69,2.80,1.45,1.12,0.89,1.12,0.89,8.50
56.35,16.26,7.76,3.57,2.62,1.89,1.05,1.15,0.63,1.05,7.66

53.79,16.19,6.47,4.57,3.47,1.74,1.95,1.37,1.00,0.74,8.73
55.63,16.28,7.87,3.72,2.48,1.99,1.40,1.19,0.70,1.08,7.65
54.09,15.76,7.96,4.70,2.77,2.21,1.27,1.27,0.66,1.11,8.19
53.79,16.19,6.47,4.57,3.47,1.74,1.95,1.37,1.00,0.74,8.73

...

This is for instance to show a dataset evolving through, for instance, time. In Gnuplot, you can then select which dataset (using it's index and the keyword, huh, index IIRC) you want to use for a given plot.

I've been using R, and up to now, I have been manually feeding it datasets one at a time, using the scan/table functions. Instead of having one big file with all the datasets in them, I have one file per dataset, and I create the tables one at a time.

Is there a (built-in, or very simple) way to read the aggregate datasets all at once, in such a way that I would have

dataset <- neatInput("my-aggregate-data")
dataset[1]    # first data set
dataset[2]    # second data set
...

or something similar?

Answer 1

I managed to consolidate the code into two lines, FWIW :)

check <- read.csv("data.csv", blank.lines.skip = F, head = F)

split(check, (cumsum(is.na(check[,1]))+1) * !is.na(check[,1]))
## $`0`
##   V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11
## 2 NA NA NA NA NA NA NA NA NA  NA  NA
## 6 NA NA NA NA NA NA NA NA NA  NA  NA

## $`1`
##      V1    V2   V3   V4   V5   V6   V7   V8   V9  V10 V11
## 1 54.32 16.17 7.42 4.28 3.09 2.11 1.66 1.22 0.99 0.82 7.9

## $`2`
##      V1    V2   V3   V4   V5   V6   V7   V8   V9  V10  V11
## 3 54.63 15.50 8.53 5.31 3.75 1.66 1.14 0.83 0.94 0.52 7.18
## 4 56.49 16.67 6.38 3.69 2.80 1.45 1.12 0.89 1.12 0.89 8.50
## 5 56.35 16.26 7.76 3.57 2.62 1.89 1.05 1.15 0.63 1.05 7.66

## $`3`
##       V1    V2   V3   V4   V5   V6   V7   V8   V9  V10  V11
## 7  53.79 16.19 6.47 4.57 3.47 1.74 1.95 1.37 1.00 0.74 8.73
## 8  55.63 16.28 7.87 3.72 2.48 1.99 1.40 1.19 0.70 1.08 7.65
## 9  54.09 15.76 7.96 4.70 2.77 2.21 1.27 1.27 0.66 1.11 8.19
## 10 53.79 16.19 6.47 4.57 3.47 1.74 1.95 1.37 1.00 0.74 8.73

Answer 2

If your third dimension is time, then it's usually best to work with specialized time/date objects. The most commonly used general-purpose time series packages in R include custom functions to do what you want. For instance, to roll-up some data originally in months to years:

> data(AirPassengers); AP = AirPassengers
> # import the package xts, which will 'auto-import' its sole dependency, 
> # the package 'zoo'
> library(xts)    

# AP is an R time series whose data points are in months
> class(AP)
[1] "ts"
> start(AP)
[1] 1949    1
> end(AP)
[1] 1960   12
> frequency(AP)
[1] 12
> AP[1:3]
[1] 112 118 132

> # step 1: convert ts object to an xts object
> X = as.xts(AP)
> class(X)
[1] "xts" "zoo"
> # step 2: create index of endpoints to pass to the aggregator function
> np = endpoints(X, on="years")
> # step 3: call the aggregator function
> X2 = period.apply(X, INDEX=np, FUN=sum)
> X2[1:3]
         [,1]
Dec 1949 1520
Dec 1950 1676
Dec 1951 2042
> # 'X2' is in years (each value is about 12X higher than the first three values for
> # AP above