views:

284

answers:

2
+1  Q: 

Matlab cell length

Ok I seem to have got the most of the problem solved, I just need an expert eye to pick my error as I am stuck.

I have a file of length [125 X 27] and I want to convert it to a file of length [144 x 27]. Now, I want to replace the missing files (time stamps) rows of zeros. (ideally its a 10 min daily average thus should have file length of 144)

Here is the code I am using:

fid = fopen('test.csv', 'rt');

data = textscan(fid, ['%s' repmat('%f',1,27)], 'HeaderLines', 1, 'Delimiter', ',');

fclose(fid);

%//Make time a datenum of the first column
time = datenum(data{1} , 'mm/dd/yyyy HH:MM')
%//Find the difference in minutes from each row

timeDiff = round(diff(datenum(time)*(24*60)))

%//the rest of the data
data = cell2mat(data(2:28));


newdata=zeros(144,27);

for n=1:length(timeDiff)

    if timeDiff(n)==10

        newdata(n,:)=data(n,:);
        newdata(n+1,:)=data(n+1,:);
    else
        p=timeDiff(n)/10
        n=n+p;

    end

end

Can somebody please help me to find the error inside my for loop. My output file seems to miss few timestamped values.

%***********************************************************************************************************

Can somebody help me to figure out the uiget to read the above file??

i am replacing

fid = fopen('test.csv', 'rt');

data = textscan(fid, ['%s' repmat('%f',1,27)], 'HeaderLines', 1, 'Delimiter', ',');

fclose(fid);

With

[c,pathc]=uigetfile({'*.txt'},'Select the file','C:\data');

file=[pathc c];

file= textscan(c, ['%s' repmat('%f',1,27)], 'HeaderLines', 1, 'Delimiter', ',');

And its not working

%


NEW ADDITION to old question

p = 1; %index into destination
for n = 1:length(timeDiff)
% if timeDiff(n) == 10
% newfile(p,:) = file(n,:);
% newfile(p+1,:)=file(n+1,:);
% p = p + 1;
% else
% p = p + (timeDiff(n)/10);
% end

q=cumsum(timeDiff(n)/10);
if q==1
newfile(p,:)=file(n,:);
p=p+1;
else
p = p + (timeDiff(n)/10);

end

end

xlswrite('testnewws11.xls',newfile);

even with the cumsum command this code fails when my file has 1,2 time stamps in middle of long missing ones example

8/16/2009 0:00 5.34
8/16/2009 0:10 3.23
8/16/2009 0:20 2.23
8/16/2009 0:30 1.23
8/16/2009 0:50 70
8/16/2009 2:00 5.23
8/16/2009 2:20 544
8/16/2009 2:30 42.23
8/16/2009 3:00 71.23
8/16/2009 3:10 3.23

My output looks like

5.34
3.23
2.23
0
0
0
0
0
0
0
0
0
5.23
544.
42.23
0
0
0
3.23

Any ideas?

+2  A: 

Update for new version of question

You seem to have misunderstood the intention of the cumsum solution I suggested. You no longer need a loop since cumsum calculates the final indeces for you. However, I left out one crucial part - the first index has to still be determined from the data file. Replace your for loop with the following:

[y,m,d] = datevec(time(1)); %# get the year, month and day corresponding to the first recorded timestamp
firstidx = time(1)-datenum(y,m,d,0,0,0)+1; %# get the offset from midnight to the first recorded timestamp
q = cumsum([firstidx ; round(diff(time)*24*60)/10]); %# these are the indeces into newdata
newdata(q,:) = data;

Previous answers

You are using n to index both newdata and data, and stopping your indexing based on length(timeDiff). This means that your loop will never touch elements of newData beyond length(timeDiff). Also, I don't at all understand what the newdata(n+1,)... line does, since it would usually get overwritten on the next iteration anyway. I think what you need is something like:

p = 1; %index into destination
for n = 1:length(timeDiff)
    if timeDiff(n) == 10
        newdata(p,:) = data(n,:);
        p = p + 1;
    else
        p = p + timeDiff(n)/10;
    end
end

You can make this look a little neater by doing:

p = cumsum(timeDiff(n)/10); % vector of destination indeces
newdata(p) = data;

(I didn't actually test this...)

Note that this scheme depends on the timeDiff vector containing integer values! You might need to throw a call to round in there too.

mtrw
Wow thanks a lot for catching that error. I did however had to add the line>newdata(p+1,:)=data(n+1,:);as my timeDiff takes diff of last two numbers so if i do not add this line it misses the last data set. With this it works perfectly. Another related question. Do you know a easy way to rewrite the headers and the timestamp( 1st column) column into the newdata file?Thanks
AP
For the header lines, I suggest you simply use `fgetl` and `fprintf`. Don't forget to remove the `HeaderLines` item from `textscan` though! For copying the timestamp, can't you print `time(i)` as you print `newdata(i)`?
mtrw
Thanks a lot. do you why i keep getting datenum faults on somefiles and not on all , though they have same formats.??? Error using ==> datenum at 174DATENUM failed.Error in ==> makefilelength144 at 18time = datenum(data{1} , 'mm/dd/yyyy HH:MM');Caused by: Error using ==> dtstr2dtnummx Failed on converting date string to date number.
AP
Try doing `disp(data{1})` - sometimes that helps spot subtle format differences.
mtrw
I got rid of 'mm/dd/yyyy HH:MM' in the time= .... and it seems to read the default format.Thanks
AP
I do however have the problem with > newdata(p+1,:)= data(n+1,:) ; as when the file is smaller , my program crashes and i do not know to include the last data pt before missing timestamps
AP
Try using the `cumsum` alternative I gave above. I think that will automatically put the last valid timestamp in the right place.
mtrw
I did not understand by reading the help of cumsum menu. So i do not know how to use that.
AP
`cumsum` is the cumulative sum. Imagine you had a data vector that should have had five elements, but was missing the third element. I think `timeDiff/10` would be `[1 1 2 1]`. `cumsum([1 1 2 1]) = [1 2 4 5]`, which tells you the correct destination indeces in `newData`.
mtrw
Tried and it failed too...Can you write the cumsum command into the above code , just to be sure i am not doing something wrong.thanks a lot
AP
@AP - it works in my testing, can you post an example timeDiff vector? I'm going to be offline for the next few hours, but can get back to this later.
mtrw
I do not know how to upload a full file . I can explain, the code fails at points like 3/3/2010 0:50; 3/3/2010 1:10; 3/3/2010 1:30; 3/3/2010 1:50; where i have 2 missing time stamps followed by a single time and then agian 2,3 time stamps missing. Please give me the code with cumsum command thanks
AP
Thanks a lot mtrw. still could not make cumsum code work. But found out the error i had . I had to recall newdata(p,:) = data(n,:); after the else command to write it properly and now its working well :)
AP
+2  A: 

For the second question:

Uigetfile returns a filename, not a file ID. Therefore, you still need to call fopen.

[c,pathc]=uigetfile({'*.txt'},'Select the file','C:\data');

file=[pathc c];

%# get a file ID
fid = fopen(file,'rt');

data= textscan(fid, ['%s' repmat('%f',1,27)], 'HeaderLines', 1, 'Delimiter', ',');

fclose(fid)
Jonas
Sweet , Thanks I still have problem on some files , it gives me error>??? Error using ==> datenum at 174DATENUM failed.Error in ==> makefilelength144 at 33time = datenum(data{1} , 'mm/dd/yyyy HH:MM');Caused by: Error using ==> dtstr2dtnummx Failed on converting date string to date number.>When i look at file it looks the same as other file which gives no error, any thoughts??
AP

related questions