skip reading headers in matlab

Question

I had a similar question. but what i am trying now is to read files in .txt format into matlab. My problem is with the headers. Manytimes due to errors the system rewrites the headers in the middle of file and then matlab cannot read the file. IS there a way to skip it? I know i can skip reading some characters if i know what the character is.

here is the code i am using.

[c,pathc]=uigetfile({'*.txt'},'Select the data','V:\data);
file=[pathc c];
data= dlmread(file, ',', 1,4);

this way i let the user pick the file. My files are huge typically [ 86400 125 ] so naturally it has 125 header fields or more depends on files.

Thanks

Because the files are so big i cannot copy , but its in format like

day time col1 col2 col3 col4 ...............................
2/3/2010 0:10 3.4 4.5 5.6 4.4 ...............................
..................................................................
..................................................................

and so on

Answer 1

With DLMREAD you can read only numeric data. It will not read date and time, as your first two columns contain. If other data are all numeric you can tell DLMREAD to skip first row and 2 columns on the right:

data = dlmread(file, ' ', 1,2); 

To import also day and time you can use IMPORTDATA instead of DLMREAD:

A = importdata(file, ' ', 1);
dt = datenum(A.textdata(2:end,1),'mm/dd/yyyy');
tm = datenum(A.textdata(2:end,2),'HH:MM');
data = A.data;

The date and time will be converted to serial numbers. You can convert them back with DATESTR function.

Answer 2

It turns out that you can still use textscan. Except that you read everything as string. Then, you attempt to convert to double. 'str2double' returns NaN for strings, and since headers are all strings, you can identify header rows as rows with all NaNs.

For example:

%# find and open file
[c,pathc]=uigetfile({'*.txt'},'Select the data','V:\data'); 
file=[pathc c];
fid = fopen(file);

%# read all text
strData = textscan(fid,'%s%s%s%s%s%s','Delimiter',','); 

%# close the file again
fclose(fid);

%# catenate, b/c textscan returns a column of cells for each column in the data
strData = cat(2,strData{:}); 

%# convert cols 3:6 to double
doubleData = str2double(strData(:,3:end));

%# find header rows. headerRows is a logical array
headerRowsL = all(isnan(doubleData),2);

%# since I guess you know what the headers are, you can just remove the header rows
dateAndTimeCell = strData(~headerRowsL,1:2);
dataArray = doubleData(~headerRowsL,:);

%# and you're ready to start working with your data