awk

Reformat a large text file into one line strings (via BASH)

File1: hello - dictionary definitions: hi hello hallo greetings salutations no more hello for you - world - dictionary definitions: universe everything the globe the biggest tree planet cess pool of organic life - I need to format this (for a huge list of words) into a term to definition format (one line per term). How can one achieve...

How to replace new lines with tab characters

hi i have pattern like below hi hello hallo greetings salutations no more hello for you i am trying to replace all new lines with tab spaces using the following command sed -e "s_/\n_/\t_g" but its not working . could nybody pls help? i need the solution in sed /awk. ...

Extracting particular column name values using sed/ awk/ perl

I have an input file say, such as: a=1 b=2 c=3 d=4 a=2 b=3 a=0 c=7 a=3 b=9 c=0 d=5 a=4 d=1 c=9 Assume that the order of column names (a,b, c and d) remains the same. How do I write a script/ command which will help me extract values specific to columns b and d? So my output should be: b=2 d=4 b=3 b=9 d=5 d=1 I could write a "not-s...

What is the difference between sed and awk ?

What is the difference between awk and sed ? What kind of application are best use cases for sed and awk tools ? ...

How can I remove selected lines with an awk script?

I'm piping a program's output through some awk commands, and I'm almost where I need to be. The command thus far is: myprogram | awk '/chk/ { if ( $12 > $13) printf("%s %d\n", $1, $12 - $13); else printf("%s %d\n", $1, $13 - $12) } ' | awk '!x[$0]++' The last bit is a poor man's uniq, which isn't available on my target. Given the...

How to detect EOF in awk?

Is there a way to determine whether the current line is the last line of the input stream? ...

How can I delete a newline if it is the last character in a file?

I have some files that I'd like to delete the last newline if it is the last character in a file. 'od -c' shows me that the command I run does write the file with a trailing new line: 0013600 n t > \n I've tried a few tricks with sed but the best I could think of isn't doing the trick: sed -e '$s/\(.*\)\n$/\1/' abc Any ideas...

awk script grouping with array

i am trying to execute awk script below on solaris but it's giving error like awk: syntax error near line 3 awk: bailing out near line 3 where is error can you help me to fix it ? Input file: $ cat details.txt Manager1|sw1 Manager3|sw5 Manager1|sw4 Manager2|sw9 Manager2|sw12 Manager1|sw2 Manager1|sw0 Output required: Group the sim...

AWK: field separator contains a '+'

echo -n 'a001~!+rr001~!+1~!+TEST DATA 1' | awk 'BEGIN {FS="~!+"} {print $2}' I have the field separator set to "~!+" and want to print the second field. AWK prints an extraneous + with rr001 as +rr001 . What am I doing wrong? ...

How can I delete all /* */ comments from a C source file?

I have a C file which I copied from somewhere else, but it has a lot of comments like below: int matrix[20]; /* generate data */ for (index = 0 ;index < 20; index++) matrix[index] = index + 1; /* print original data */ for (index = 0; index < 5 ;index++) How can I delete all the comments enclosed by /* and */. Sometimes, the comment...

Bulk Insert Code Before </body> Tag in 100 Files

I'd like to insert <?php include_once('google_analytics.php'); ?> before the closing body tag of about 100 php files. Unfortunately the person who made the site didn't make a header or footer template. What is the best way to do this? I've tried using grep/find for getting a list of files and piping the results through xargs to se...

how to use sed, awk, or gawk to print only what is matched?

I see lots of examples and man pages on how to do things like search-and-replace using sed, awk, or gawk. But in my case, I have a regular expression that I want to run against a text file to extract a specific value. I don't want to do search-and-replace. This is being called from bash. Let's use an example: Example regular express...

Using Unix Tools to Extract String Values

I wrote a small Perl script to extract all the values from a JSON formatted string for a given key name (shown below). So, if I set a command line switch for the Perl script to id, then it would return 1,2, and stringVal from the JSON example below. This script does the job, but I want to see how others would solve this same problem usi...

Awk - print next record following matched record

I'm trying to get a next field after matching field using awk. Is there an option to do that or do I need to scan the record into array then check each field in array and print the one after that? What I have so far: The file format is: <FIELD><separator "1"><VALUE><separator "1"><FIELD><separator "1"><VALUE> ... and so on, field|v...

awk - how to specify field separator as binary value 0x1

Is it possible to specify the separator field FS in binary for awk? I have data file with ascii data fields but separated by binary delimiter 0x1. If it was character '1' it would look like this: awk -F1 '/FIELD/ { print $1 }' Or in script: #!/bin/awk -f BEGIN { FS = "1" } /FIELD/ { print $1 } How can I specify FS/F to be 0x1. ...

help with selecting rows using awk

I have a file as follows 2.54 Ghz val 2.53 Ghz val1 1.6 Ghz val2 800 Mhz val3 2.54 Ghz val4 2.53 Ghz val5 1.6 Ghz val6 800 Mhz val7 and the pattern continues ... I want to extract all 2.54 Ghz values in one file1 and all 2.53 Ghz values in another file2 , 1.60 Ghz values in file3 and 800 Mhz values in file4 ...

How can I change spaces to underscores and lowercase everything?

I have a text file which contains: Cycle code Cycle month Cycle year Event type ID Event ID Network start time I want to change this text so that when ever there is a space, I want to replace it with a _. And after that, I want the characters to lower case letter like below: cycle_code cycle_month cycle_year event_type_id event_id ne...

How can I remove responses from LiveHTTPHeaders output using awk, perl or sed?

Let's say I have something like this (this is only an example, actual request will be different: I loaded StackOverflow with LiveHTTPHeaders enabled to have some samples to work on): http://stackoverflow.com/ GET / HTTP/1.1 Host: stackoverflow.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.2) Gecko/20070220 Firefox/2...

How to read files with different encodings using Awk?

How can I correctly read files in encodings other than UTF8 in Awk? I have a file in Hebrew/Windows-1255 encoding. A simple {print $0} awk prints stuff like �. how can I make it read correctly? ...

printing variable inside awk

In this script , I want awk to print the variables $file, $f, $order and sum/NR (all in a single row) #!/bin/bash for file in pmb_mpi tau xhpl mpi_tile_io fftw ; do for f in 2.54 1.60 800 ;do if [ ${f} = 2.54 ] then for order in even odd ; do # echo ${file}_${f}_${order}_v1.xls >> P-state-summary.xls awk '{s...