gunzip preform task and rezip

Question

Hello all

I want to perform this

awk -F, '$1 ~ /F$/' file.dat

on a whole direcory of gziped files

I want to be able to loop through each file unzip it perform the above command (print out any findings) rezip and move onto the next zipped file

how can this be done?

Thanks

Answer 1

how about a shell script that does:

gunzip $1.gz
awk -F, '$1 ~ /F$/' $1
gzip $1

Just make sure $1 doesn't have the .gz extension.

then you can do something like:

find -name \*.gz -type f|xargs my_shell_script

Answer 2

Well if you have to have it all in one line:

find . -name \*.gz -print0 | xargs -0 gunzip -c | awk -F, '$1 ~ /F$/'

That should work for you

Answer 3

It looks to me that you want to output lines where the first field, in a comma-separated list of fields, ends with 'F'

If you don't care about listing which .gz file the lines come from, or which file within the gzipped file -- that is, you just want the lines listed -- you don't even have to gunzip the .gz files first, that way you won't have to re-zip them.

zcat file.gz | awk -F, '$1 ~ /F$/'

For every file in the current directory tree, use a find with xargs. This example limits it to only the current directory, but just leave out the "-maxdepth 1" to get the whole directory tree

find . -maxdepth 1 -name \*.gz -print0 | xargs -0 zcat | awk -F, '$1 ~ /F$/'

This says to find files that end in ".gz" and write their names with a NUL terminator (the 0 in -print0), pipe that output through xargs which will pick the list apart based on NUL chacracters (the "-0" arg) and run zcat on the files. Pipe that output through your awk command and you'll get the relevent lines written out.