How to grep all the data in a specified col from standard output?

Question

Now i have a standard output which is printed by my program and displayed on the screen. The outputs looks like the following part:

Parsing command line string 'InputFile = foreman.qcif'.
Parsing command line string 'NumberReferenceFrames = 1'.
Parsing command line string 'QPISlice = 24'.
Parsing command line string 'QPPSlice = 24'.

------------------------------- JM 11.0 (FRExt) --------------------------------
 Input YUV file                    : foreman.qcif 
 Output H.264 bitstream            : test.264 
 Output YUV file                   : test_rec.yuv 
 YUV Format                        : YUV 4:2:0 
 Frames to be encoded I-P/B        : 150/0
 PicInterlace / MbInterlace        : 0/0
 Transform8x8Mode                  : 1
-------------------------------------------------------------------------------
  Frame  Bit/pic    QP   SnrY    SnrU    SnrV    Time(ms) MET(ms) Frm/Fld Ref  
-------------------------------------------------------------------------------
0000(NVB)     168 
0000(IDR)   34280   24  39.724  41.720  43.998       286       0    FRM    1
0002(P)      7432   24  38.857  41.565  43.829       402      99    FRM    1
0004(P)      8976   24  38.642  41.275  43.698       409      97    FRM    1
0006(P)      8344   24  38.427  41.266  43.515       407      99    FRM    1
0008(P)      8224   24  38.609  41.082  43.524       413      94    FRM    1
0010(P)      7784   24  38.655  40.991  43.235       406      95    FRM    1
0012(P)      7136   24  38.534  40.687  43.273       411      95    FRM    1
0014(P)      6688   24  38.464  40.756  43.146       410      92    FRM    1
0016(P)      7720   24  38.516  40.585  42.851       410      91    FRM    1
0018(P)      6864   24  38.474  40.631  42.958       411     101    FRM    1
0020(P)      8392   24  38.433  40.607  42.646       415      99    FRM    1
0022(P)      9744   24  38.371  40.554  42.498       416      94    FRM    1
0024(P)      8368   24  38.362  40.531  42.380       417      93    FRM    1
0026(P)      7904   24  38.414  40.586  42.415       418      95    FRM    1
0028(P)      8688   24  38.403  40.523  42.366       418      96    FRM    1
0030(P)      9128   24  38.545  40.390  42.661       416      89    FRM    1
0032(P)      9664   24  38.399  40.538  42.740       413      88    FRM    1
0034(P)      8928   24  38.394  40.590  42.852       414      95    FRM    1
0036(P)     10024   24  38.423  40.562  42.697       415      92    FRM    1
0038(P)      9320   24  38.442  40.389  42.689       414      94    FRM    1
0040(P)      7304   24  38.404  40.487  42.884       410      90    FRM    1
0042(P)      8560   24  38.447  40.590  42.673       411      95    FRM    1

.......

Now I only need to process the 4th col of SnrY.

So, my question is how to grep this rol and to store them in a data.txt file?

Then I can use plot tool or Matlab to plot the data trends with these data..

Any advices? Many thanks for your kind help!

Addition:

Since it needs to pick those data from standard output, do I need to use(add) the command provided by you in the command line to make it works during outputs those data ?

Answer 1

$ awk '/FRM/ { print $4 }' < in.txt > data.txt

Note: I'm just guessing that the interesting lines contain FRM. If this is not the case, you must come up with a different way to identify them.

Answer 2

This will match numbers like 38.4* in the fourth column: grep -E '^([^ ]+[ ]+){3}38.4'

Answer 3

./cmd_that_generates_results | awk '/^[0-9]/{print $4}' > data.txt

Quick breakdown of awk statement:

• /^[0-9]/ : match lines that start with a number
• {print $4} : print out fourth column

(Updated): to address question "Since it needs to pick those data from standard output, do I need to use(add) the command provided by you in the command line to make it works during outputs those data ?"

You can use the commands given in answers by piping to them the standard output of your program using the pipe (|) command. The results can then be store into "data.txt" using >.

see example above. Similar techniques can be used for other solutions in this page.

Answer 4

tail -n+17 in.txt | cut -c 23-31

Answer 5

$ ./command | awk 'BEGIN{RS="--+";FS="\n"} END{ for(i=1;i<=NF;i++){if($i ~ /^[0-9]/){m=split($i,a," ");if(a[4]) print a[4]}}}' 
39.724
38.857
38.642
38.427
38.609
38.655
38.534
38.464
38.516
38.474
38.433
38.371
38.362
38.414
38.403
38.545
38.399
38.394
38.423
38.442
38.404
38.447

I am anticipating that the data you want to get

1. need not be after the 16th line, may be after the 17th , or 18th ,etc
2. or that FRM/Fld field may have different indicators,
3. or the 4th field may not be at the exact character position every time.

So the above just say: set record separator as the line with dashes "-" , so the last record is the whole data you need. then using the newline as field separator (FS="\n") , each field will be each line of data. Split them up with spaces and get element 4 of the splitted line. that will be your desired column.