SAS - Reading a File Backwards?

Question

I need SAS to read many large log files, which are set up to have the most recent activities at the bottom. All I need is the most recent time a particular activity occurred, and I was wondering if it's possible for SAS to skip parsing the (long) beginning parts of the file.

I looked online and found how to read a dataset backwards, but that would require SAS to first parse everything in the .log file into the dataset first. Is it possible to directly read the file starting from the very end so that I can stop the data step as soon as I find the most recent activity of a particular type?

I read up on infile as well, and the firstobs option, but I have no idea how long these log files are until they are parsed, right? Sounds like a catch-22 to me. So is what I'm describing doable?

Thanks for any input!

Answer 1

I'd probably set up a filename pipe statement to use an operating system command like tail -r or tac to present the file in reverse order to SAS. That way SAS can read the file normally and you don't have to worry about how long the file is.

Answer 2

If you mean parsing a sas log file, I am not sure if reading the log file backward is worth the trouble in practice. For instance, the following code executes less than a tenth of a second on my PC and it is writing and reading a 10,000 line log file. How big is your log files and how many are there? Also as shown below, you don't have to "parse" everything on every line. You can selectively read some parts of the line and if it is not what you are looking for, then you can just go to the next line.

%let pwd = %sysfunc(pathname(WORK));
%put pwd=&pwd;
x cd &pwd;

/* test file. more than 10,000 line log file */
data _null_;
  file "test.log";
  do i = 1 to 1e4;
    r = ranuni(0);
    put r binary64.;
    if r < 0.001 then put "NOTE: not me!";
  end;
  put "NOTE: find me!";
  do until (r<0.1);
    r = ranuni(0);
    put r binary64.;
  end; 
  stop;
run;

/* find the last line that starts with
   NOTE: and get the rest of the line. */
data _null_;
  length msg $80;
  retain msg;
  infile "test.log" lrecl=80 eof=eof truncover;
  input head $char5. @;
  if head = "NOTE:" then input @6 msg $char80.;
  else input;
  return;
eof: 
  put "last note was on line: " _n_ ;
  put "and msg was: " msg $80.;
run;
/* on log
   last note was on line: 10013
   and msg was:  find me!
*/