Reading a text file in MATLAB line by line

Question

Hi, I have a CSV file, I want to read this file and do some pre-calculations on each row to see for example that row is useful for me or not and if yes I save it to a new CSV file. can someone give me an example? in more details this is how my data looks like: (string,float,float) the numbers are coordinates.

ABC,51.9358183333333,4.183255
ABC,51.9353866666667,4.1841
ABC,51.9351716666667,4.184565
ABC,51.9343083333333,4.186425
ABC,51.9343083333333,4.186425
ABC,51.9340916666667,4.18688333333333

basically i want to save the rows that have for distances more than 50 or 50 in a new file.the string field should also be copied. thanks

Answer 1

Hi, here is the doc to read a csv : http://www.mathworks.com/access/helpdesk/help/techdoc/ref/csvread.html and to write : http://www.mathworks.com/access/helpdesk/help/techdoc/ref/csvwrite.html

EDIT

An example that works :

file.csv :

1,50,4.1
2,49,4.2
3,30,4.1
4,71,4.9
5,51,4.5
6,61,4.1

the code :

File = csvread('file.csv')
[m,n] = size(File)
index=1
temp=0
for i = 1:m
    if (File(i,2)>=50)
        temp = temp + 1
    end
end
Matrix = zeros(temp, 3)

for j = 1:m
    if (File(j,2)>=50)
        Matrix(index,1) = File(j,1)
        Matrix(index,2) = File(j,2)
        Matrix(index,3) = File(j,3)
        index = index + 1
    end
end
csvwrite('outputFile.csv',Matrix)

and the output file result :

1,50,4.1
4,71,4.9
5,51,4.5
6,61,4.1

This isn't probably the best solution but it works! We can read the CSV file, control the distance of each row and save it in a new file.

Hope it will help!

Answer 2

If you really want to process your file line by line, a solution might be to use fgetl:

1. Open the data file with fopen
2. Read the next line into a character array using fgetl
3. Retreive the data you need using sscanf on the character array you just read
4. Perform any relevant test
5. Output what you want to another file
6. Back to point 2 if you haven't reached the end of your file.

Unlike the previous answer, this is not very much in the style of Matlab but it might be more efficient on very large files.

Hope this will help.

Answer 3

You cannot read text strings with csvread. Here is another solution:

fid1 = fopen('test.csv','r'); %# open csv file for reading
fid2 = fopen('new.csv','w'); %# open new csv file
while ~feof(fid)
    line = fgets(fid1); %# read line by line
    A = sscanf(line,'%*[^,],%f,%f'); %# sscanf can read only numeric data :(
    if A(2)<4.185 %# test the values
        fprintf(fid2,'%s',line) %# write the line to the new file
    end
end
fclose(fid1);
fclose(fid2);

Answer 4

You could actually use XLSREAD to accomplish this. After first placing your sample data above in a file 'input_file.csv', here is an example for how you can get the numeric values, text values, and the raw data in the file from the three outputs from XLSREAD:

>> [numData,textData,rawData] = xlsread('input_file.csv')

numData =     %# An array of the numeric values from the file

   51.9358    4.1833
   51.9354    4.1841
   51.9352    4.1846
   51.9343    4.1864
   51.9343    4.1864
   51.9341    4.1869


textData =    %# A cell array of strings for the text values from the file

    'ABC'
    'ABC'
    'ABC'
    'ABC'
    'ABC'
    'ABC'


rawData =     %# All the data from the file (numeric and text) in a cell array

    'ABC'    [51.9358]    [4.1833]
    'ABC'    [51.9354]    [4.1841]
    'ABC'    [51.9352]    [4.1846]
    'ABC'    [51.9343]    [4.1864]
    'ABC'    [51.9343]    [4.1864]
    'ABC'    [51.9341]    [4.1869]

You can then perform whatever processing you need to on the numeric data, then resave a subset of the rows of data to a new file using XLSWRITE. Here's an example:

index = sqrt(sum(numData.^2,2)) >= 50;  %# Find the rows where the point is
                                        %#   at a distance of 50 or greater
                                        %#   from the origin
xlswrite('output_file.csv',rawData(index,:));  %# Write those rows to a new file

Answer 5

Just read it in to MATLAB in one block

fid = fopen('file.csv');
data=textscan(fid,'%s %f %f','delimiter',',');
fclose(fid);

You can then process it using logical addressing

ind50 = data{2}>=50 ;

ind50 is then an index of the rows where column 2 is greater than 50. So

data{1}(ind50)

will list all the strings for the rows of interest. Then just use fprintf to write out your data to the new file