gawk

Can awk skip files which do not exist, race-free?

Is there a way to make awk (gawk) ignore or skip missing files? That is, files passed on the command line that no longer exist in the file system (e.g. rapidly appearing/disappearing files under /proc/[1-9]*). By default, a missing file is a fatal error :-( I would like to be able to do the equivalent of something like this: BEGIN { M...

AWK: redirecting script output from script to another file with dynamic name

Hi all, I know I can redirect awk's print output to another file from within a script, like this: awk '{print $0 >> "anotherfile" }' 2procfile (I know that's dummy example, but it's just an example...) But what I need is to redirect output to another file, which has a dynamic name like this awk -v MYVAR"somedinamicdata" '{print $0 ...

In sed or awk, how do I handle record separators which *may* span multiple lines?

My log file is: Wed Nov 12 blah blah blah blah cat1 Wed Nov 12 blah blah blah blah Wed Nov 12 blah blah blah blah Wed Nov 12 blah blah blah blah cat2 more blah blah even more blah blah Wed Nov 12 blah blah blah blah cat3 Wed Nov 12 blah blah blah blah cat4 I want to parse out the full multiline entries where cat is fo...

parse csv file using gawk

How do you parse a csv file using gawk? Simply setting FS="," is not enough, as a quoted field with a comma inside will be treated as multiple fields. Example using FS="," which does not work: file contents: one,two,"three, four",five "six, seven",eight,"nine" gawk script: BEGIN { FS="," } { for (i=1; i<=NF; i++) printf "field #...

Awk matching of entire record using regular expression

Using Awk I want to match the entire record using a regular expression. By default the regular expression matching is for parts of a record. The ideal solution would: Be general for all fields, regardless of the field separator used. Not treat the entire input as a single field and parse it manually using string functions. Work in a g...

AWK - My regexp won't respect case.

I'm running Ubuntu 8.04 and my code looks like this... for (i=1;i<=n;i++) { if (arr[i] ~ /^[A-Z]{2,4}$/) printf(arr[i]) } I quickly discovered that the {n} expression won't work in gawk without the --posix switch. Once enabled the expression works but it is case-insenitive matching AAAA and aaaa. What is going on here? ...

Best reference / crib sheet for AWK

In a series of similar questions, what is the best AWK reference you've ever seen? If there isn't really one (I've yet to find the grail), perhaps we could compile one in a separate question. ...

How to perform calculation over a log file

I have a that looks like this: I, [2009-03-04T15:03:25.502546 #17925] INFO -- : [8541, 931, 0, 0] I, [2009-03-04T15:03:26.094855 #17925] INFO -- : [8545, 6678, 0, 0] I, [2009-03-04T15:03:26.353079 #17925] INFO -- : [5448, 1598, 185, 0] I, [2009-03-04T15:03:26.360148 #17925] INFO -- : [8555, 1747, 0, 0] I, [2009-03-04T15:03:26.367523...

Printing thousand seperated floats with GAWK

I must process some huge file with gawk. My main problem is that I have to print some floats using thousand separators. E.g.: 10000 should appear as 10.000 and 10000,01 as 10.000,01 in the output. I (and Google) come up with this function, but this fails for floats: function commas(n) { gsub(/,/,"",n) point = index(n,".") - 1 if ...

Anyone know how to make a self-contained Awk/Gawk program on Windows

I'm using an awk script to do some reasonably heavy parsing that could be useful to repeat in the future but I'm not sure if my unix-unfriendly co-workers will be willing to install awk/gawk in order to do the parsing. Is there a way to create a self-contained executable from my script? ...

fatal: function name previously defined

For the listing below, I get an error: fatal: function name `myprint' previously defined $3 > 0 { myprint ($3) } function myprint(num) { printf "%6.3g\n", num } ...

Scanning all elements of an array in GAWK returns numbers instead of values

Given the following function: function process_pipes(text) { split(text,recs,"|"); for (field in recs){ printf ("|%s|\n", field) } } If the input is: 0987654321|57300|ERROR account number not found|GDUMARESQ|0199|9|N|0|| Why do I get the numbers below instead of the text? |4| |5| |6| |7| |8| |9| |10| |1| |2| |3|...

A regex for a class definition in java

The problem is as follows: There is a massive codebase in Java (hundreds of files in tens of packages) where we need the ability to add and remove the keyword strictfp in each class definition. I am planning to use either sed or awk to perform this substitution. However, I would like to avoid the word "class" in comments or elsewhere fr...

using gawk in a shell script

Hi I want to use gawk in a for loop. Something like this: for i in gawk {print $1} | tr '\n' ' ' do something using $i this isn't working of course. Ideas? ...

Using Awk to process a file where each record has different fixed-width fields.

I have some data files from a legacy system that I would like to process using Awk. Each file consists of a list of records. There are several different record types and each record type has a different set of fixed-width fields (there is no field separator character). The first two characters of the record indicate the type, from thi...

using gawk in a shell script

I want to do something of the sort: for i in 1 2 3 do gawk '{if ($i==$2) {print $0;}}' filename done is this possible? thanks ...

How could I print an array slice in awk?

In a file I am reformatting, I would like to put the last column as the first and have the rest of the columns stay the same. I could do this easily in python, but thought I'd learn some awk this evening. Here is an example: (before) polk_describes:1 diatribe_that:1 #label#:negative (after) #label#:negative polk_describes:1 diatri...

Format of File using Awk in PowerShell

All I did was ls to a file, then ran a simple awk print. I'm new to both PowerShell and Awk, but the output is obsviously not what's expected. Can anyone explain this? Does it have something to do with the format of the file? PS C:\Documents and Settings\lmoser\My Documents\Test> awk '{ print }' lsfiles.txt > awkedlsfile.txt PS C:\Do...

GNU awk: accessing captured groups in replacement text

This seems like it should be dirt simple, but the awk gensub/gsub/sub behavior has always been unclear to me, and now I just can't get it to do what the documentation says it should do (and what experience with a zillion other similar tools suggests should work). Specifically, I want to access "captured groups" from a regex in the repla...

Formula in gawk

I have a problem that I’m trying to work out in gawk. This should be so simple, but my attempts ended up with a divide by zero error. What I trying to accomplish is as follows – maxlines = 22 (fixed value) maxnumber = > max lines (unknown value) Example: maxlines=22 maxnumber=60 My output should look like the following: print li...