awk

Using AWK to treat files

I have something to do that must be finished before 4.00PM. I want to create a batch file with awk, grep or sed that keeps all lines beginning with 'INSERT' and deletes the other lines. After this, I want to replace a string "change)" by "servicechange)" when the 3rd word in the treated line is "donextsit". Please explain how to d...

Awk to grab colo(u)r codes from CSS files aka School me in Awk

Nice and (hopefully) easy. I am trying to work out how to grab the variable #XXX from a text file (css file) containing strings like hr { margin: 18px 0 17px; border-color: #ccc; } h1 a:hover, h2 a:hover, h3 a:hover { color: #001100; } Which I would like to return as ccc 001100 The plan then is to throw this throug...

Substring extraction using bash shell scripting and awk

So, I have a file called 'dummy' which contains the string: "There is 100% packet loss at node 1". I also have a small script that I want to use to grab the percentage from this file. The script is below. result=`grep 'packet loss' dummy` | awk '{ first=match($0,"[0-9]+%") last=match($0," packet loss") s=substr($0,fi...

linux ps aux command

How obtain the running process with absolute path name from ps command 13598 ? Ssl 0:00 /opt/something --run-id 2 -n /home/laks/f1 -l l.log I need the output as - /opt/something --run-id 2 -n /home/laks/f1 -l l.log ...

Use Awk to Print every character as its own column?

Hi there, I am in need of reorganizing a large CSV file. The first column, which is currently a 6 digit number needs to be split up, using commas as the field separator. For example, I need this: 022250,10:50 AM,274,22,50 022255,11:55 AM,275,22,55 turned into this: 0,2,2,2,5,0,10:50 AM,274,22,50 0,2,2,2,5,5,11:55 AM,275,22,55 Let ...

How to use a variable in an awk expression

I'm trying to make this command: sed bla bla filename | awk '{printf "%s %s_entry_%.3f %.3f %.3f %.3f",$1,$3,$4,$5,$6,$7}' But the thing is, i want the %.3f part to be variable. So in one case it could be %.3f and in another it could be %.3f %.3f %.3f. So i'll just use a static one in my example code for clarity. So if i want 4 of the...

Using for or while loops

Every month, 4 or 5 text files are created. The data in the files is pulled into MS Access and used in a mailmerge. Each file contains a header. This is an example: HEADER|0000000130|0000527350|0000171250|0000058000|0000756600|0000814753|0000819455|100106 The 2nd field is the number of records contained in the file (excluding the hea...

What can I do using awk that I cannot do in Perl?

I had read somewhere about one specific feature that is present in awk but not in Perl. I have failed in locating it again. I would appreciate it if anyone here can point it out. This might be a useless trivia, but I am still curious to know. ...

awk - Remove line if field is duplicate

Looking for an awk (or sed) one-liner to remove lines from the output if the first field is a duplicate. An example for removing duplicate lines I've seen is: awk 'a !~ $0; {a=$0}' Tried using it for a basis with no luck (I thought changing the $0's to $1's would do the trick, but didn't seem to work). ...

awk or perl file editing & manipulation

I have a standard passwd file & a usermap file - which maps unix name (eg jbloggs) with AD account name (eg bloggsjoe) in the format: jbloggs bloggsjoe jsmith smithjohn ... etc. How can I edit the passwd file to swap the original unix name with the AD account name so each line of the passwd file has the AD account name instead. A...

How to use ":" as awk field separator?

Given following command: echo "1: " | awk '/1/ -F ":" {print $1}' why does awk output: 1: ...

[sed] Multiline trimming

I have a html file that I want to trim. I want to remove a section from the beginning all the way to a given string, and from another string to the end. How do I do that, preferably using sed? ...

AWK If/ElseConditional Problem

I have a data that looks like this: foo foo scaffold_7 1 4845 6422 4845 bar bar scaffold_7 -1 14689 16310 16310 What I want to do is to process the above lines where I just want to print column 1,2,3, 7 and one more column after 7th. But with condition when printing column 7 onwards. Below is my awk script: awk ...

Creating multiple csv files from data within a csv file.

System OSX or Linux I'm trying to automate my work flow at work, each week I receive an excel file, which I convert to a csv. An example is: ,,L1,,,L2,,,L3,,,L4,,,L5,,,L6,,,L7,,,L8,,,L9,,,L10,,,L11, Title,r/t,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,neede d,actual,Inst,needed,actual,Inst,needed,actua...

Swapping of columns in a file and remove duplicates

Hi all i have a file like this: term1 term2 term3 term4 term2 term1 term5 term3 ..... ..... what i need to do is to remove duplicates in any order they appear, such as: term1 term2 and term2 term1 is a duplicate to me. It is a really long file, so I'm not sure what can be faster. Does anyone has an idea on how to do this? awk...

Awk: print all other columns but not $1, $2 and $3

Too cumbersome: awk '{print " "$4" "$5" "$6" "$7" "$8" "$9" "$10" "$11" "$12" "$13}' things ...

OSX, G/AWK, Bash - "illegal statement, unterminated string" and no file output.

I have a script that somebody from SO kindly provided to solve an issue I was having, However, I'm having some issues getting it to work on OSX. gawk --version GNU Awk 3.1.6 awk --version awk version 20100208 The original source is: awk -F, -vOFS=, -vc=1 ' NR == 1 { for (i=1; i<NF; i++) { if ($i != "") { g[c]...

Getting ID of an instance newly launched with ec2-api-tools

I'm launching an EC2 instance, by invoking ec2-run-instances from simple a bash script, and want to perform further operations on that instance (e.g. associate elastic IP), for which I need the instance id. The command is something like ec2-run-instances ami-dd8ea5a9 -K pk.pem -C cert.pem --region eu-west-1 -t c1.medium -n 1, and its o...

How do I delete the first line in a file?

I can't search for a particular string, since they're all very similar, but I'd like something simple to chop out the first 4 lines in a file. They're all variable length too. I've had a a think about perl, and it all seems harder than I thought, but I'd like to do it in Perl, AWK or a shell command if possible. Does anybody have a si...

Parsing the first column of a csv file to a new file.

Operating System: OSX Method: From the command line, so using sed, cut, gawk, although preferably no installing modules. Essentially I am trying to take the first column of a csv file and parse it to a new file. Example input file EXAMPLEfoo,60,6 EXAMPLEbar,30,6 EXAMPLE1,60,3 EXAMPLE2,120,6 EXAMPLE3,60,6 EXAMPLE4,30,6 Desire output...