Read in multiple .txt files with header in R

Question

Okay, I'm trying to use this method to get my data into R, but I keep on getting the error:

  Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,  : 
  line 1 did not have 22 elements

This is the script that I'm running:

library(foreign)

setwd("/Library/A_Intel/")

filelist <-list.files()

#assuming tab separated values with a header    
 datalist = lapply(filelist, function(xx)read.table(xx, header=T, sep=";")) 

#assuming the same header/columns for all files
 datafr = do.call("rbind", datalist) 

Keep in mind, my priorities are:

1. To read from a .txt file
2. To associate the headers with the content
3. To read from multiple files.

Thanks!!!

Answer 1

It appears that one of the files you are trying to read does have the same number of columns as the header. To read this file, you may have to alter the header of this file, or use a more appropriate column separator. To see which file is causing the problem, try something like:

datalist <- list()
for(filename in filelist){
  cat(filename,'\n')
  datalist[[filename]] <- read.table(filename, header = TRUE, sep = ';')
}

Another option is to get the contents of the file and the header separately:

datalist[[filename]] <- read.table(filename, header = FALSE, sep = ';')
thisHeader <- readLines(filename, n=1)
## ... separate columns of thisHeader ...
colnames(datalist[[filename]]) <- processedHeader

If you can't get read.table to work, you can always fall back on readLines and extract the file contents manually (using, for example, strsplit).

Answer 2

To keep in the spirit of avoiding for loops an initial sanity check before loading all of the data could be done with

lapply(filelist, function(xx){
       print (scan(xx, what = 'character', sep=";", nlines = 1))} )

(assuming your header is separated with ';' which may not be the case)