gawk

how to use sed, awk, or gawk to print only what is matched?

I see lots of examples and man pages on how to do things like search-and-replace using sed, awk, or gawk. But in my case, I have a regular expression that I want to run against a text file to extract a specific value. I don't want to do search-and-replace. This is being called from bash. Let's use an example: Example regular express...

print second last column/field in AWK

I want to print the second last column or field in awk. the number of fields is variable. I know that I should be able to use $NF but not sure how it can be used. And this does not seem to work awk ' { print ( $NF-- ) } ' ...

gawk / awk: piping date to getline *sometimes* won't work

Hi! I'm attempting to convert dates from one format to another: From e.g. "October 29, 2005" to 2005-10-29. I have a list of 625 dates. I use Awk. The conversion works -- most of the time. Hovewer, sometimes the conversion won't happen at all, and the variable supposed to hold the (converted) date remains undefined. This always happen...

Calling gawk from Python

I am trying to call gawk (the GNU implementation of AWK) from Python in this manner. import os import string import codecs ligand_file=open( "2WTKA_ab.txt", "r" ) #Open the receptor.txt file ligand_lines=ligand_file.readlines() # Read all the lines into the array ligand_lines=map( string.strip, ligand_lines ) ligand_file.close() for ...

How to use ":" as awk field separator?

Given following command: echo "1: " | awk '/1/ -F ":" {print $1}' why does awk output: 1: ...

Efficient splitting of elements in a field

I have a field in a text file exported from a database. The field contains addresses but sometimes they are quite long and the database allows them to contain multiple lines. When exported, the newline character gets replaced with a dollar sign like this: first part of very long address$second part of very long address$third part of v...

OSX, G/AWK, Bash - "illegal statement, unterminated string" and no file output.

I have a script that somebody from SO kindly provided to solve an issue I was having, However, I'm having some issues getting it to work on OSX. gawk --version GNU Awk 3.1.6 awk --version awk version 20100208 The original source is: awk -F, -vOFS=, -vc=1 ' NR == 1 { for (i=1; i<NF; i++) { if ($i != "") { g[c]...

unwanted quote marks in output from printf

I have some address data which has been exported from a database. If the address had multiple lines, the exported data has joined all the lines into one string with the former lines being separated by dollars signs. Here's one of the addresses: INFORMATION DELIVERY DEPT$704 CHERRY ST$ATLANTA, GA 30332-0900 I'm splitting this into an ...

Using awk to print all columns from the nth to the last

right now I have this line, and it worked until I had whitespace in the second field. svn status | grep '\!' | gawk '{print $2;}' > removedProjs is there a way to have awk print everything in $2 or greater? ($3, $4.. until we don't have anymore columns?) I suppose I should add that I'm doing this in a windows environment with cygwin...

matching files with regular expressions

Dear all, I have an input file with a list of movies (Note that there might be some repeated entries): American_beauty__1h56mn38s_ As_Good_As_It_Gets As_Good_As_It_Gets _DivX-ITA__Casablanca_M_CURTIZ_1942_Bogart-bergman_ Capote_EN_DVDRiP_XViD-GeT-AW _DivX-ITA__Casablanca_M_CURTIZ_1942_Bogart-bergman_ I would to find the corresponding...

changing content of the column without destroying formating

Hi, $ echo "a b" | awk '{print $0; $1="1"; print $0}' a b 1 b I would like to receive formatted output like this: a b 1 b Is there easy way to do that (without IFS,OFS changes)? I am changing columns in big table and afterwards it looks ugly. I don't want to reformat each column. Thanks. ...

calling gawk from an R script

Hi i am using R on windows XP i have cygwin on my shell path what i want to do is send a command to gawk via R shell command this way: shell("gawk "{print $1}"", m[1],"_", h[i]."_79.7.dat""} i get this error Error: unexpected '{' in "shell("gawk "{" how can i fix this problem? Thank you ...

awk replacing text but not for the last line.

i tried the awk liner below(on windows command-prompt):not working properly gawk -v var="hot" "{ if(!NR){gsub(/cool/,var,$0) ;print} else{print}}" awk_test input file is below this is a cool jack this nota cool kack this obviously a cool jack a unix solution is also feasible ...

using 'set' variables that are gawk one liners

i am writing a short bat file that contours a xyz file with GMT utilities (generic mapping tool) i want to read the max and min file and use it later in the bat file what i did is set max_color=gawk "BEGIN {max = 0} {if ($3>max) max=$3} END {print max}" %file% set min_color=gawk "BEGIN {min = %max_color%} {if ($3'<'min) min=$3} ...

Shell variable interpreted wrongly in awk

Hi, In following code I am trying to pass shell varibale to awk. But when I try to run it as a.sh foo_bar the output printed is "foo is not declared" and when I run it as a.sh bar_bar the output printed is " foo is declared" . Is there a bug in awk or I am doing something wrong here? I am using gawk-3.0.3. #!/bin/awk model=$1 awk...

Substitute Array Elements into Regular Expression Statements / Variable Names

Hi All, Been wondering if this is doable in AWK for some time but always worked around it in the past. Below I initialize an array with 3 months of the year... for readability I ommited the other 9 months. These months are then used in the if-statement as part of a regular expression, but AWK doesn't like it. I can't seem to find any...

TCP Networking in Gawk Works for Some Addresses but not Others

Hi All, I've been fiddling with TCP/IP networking in Gawk and am having a hard time figuring out why it behaves well with some sites but not for others. I've even tried using HTTP Live Headers in Windows to try and debug what's going on, but to no avail. The sample Gawk code below (Version 3.1.5) will work fine for the site www.sobell...

awk return parent HTML tag value if its child tag content is matched - possible?

Hello, I've been searching for solution to this problem for quite some time, but I can't figure it out on my own. So I have bunch of HTML blocks of code, and I want to search for specific string that is contained in one of the inner tags and if there's match I want return it's parent tag value. Here's example" <li rel="Returns this val...

How to convert time format from "2010-10-08 00:00:01" to "1286467201" in awk

In awk, Is there a way to convert the time format from "2010-10-08 00:00:01" to 1286467201 as using the command "date" $ date +%s -d '2010-10-08 00:00:01' 1286467201 ...

AWK program to find the average rainfall of three states

I want to find the average rainfall of any three states say CA, TX and AX for a particular month from Jan to Dec . Given input file delimited by TAB SPACES and has the format city name, the state , and then average rainfall amounts from January through December, and then an annual average for all months. EG may look like AVOCA PA 3...