awk

Applying Unix's Head on AWK Through Find Command

I want to output top 10 lines of AWK command in the list of files given by find, using this snippet: $ find . -name "*.txt" -print -exec awk '$9 != ""' \| head -n10 {} \; Note also that I want to print out the file names being processed. But why I get such error: awk: cmd. line:2: fatal: cannot open file `|' for reading (No such fi...

How can I generate a file of random negative and positive integers in serial?

I want a file of randomly generated positive or negative serial integers. For now, I ask the file contain roughly (no guarantee required) equal numbers of negative and positive, but make it easy to change the proporotions later. By "serial", I mean the kth random negative is equal to -k, and the kth random positive is equal to +k. Thi...

How can you run AWK in a Bash script?

The problem is related to my answer here. I cannot get the following code to work #!/bin/bash wget http://www.google.com/coop/cse?cx=017685108214920485934%3Aizx6pzojwka -O CSE-spanien awk '/^searches [1-9]* sites, including:/ { print $2 }' CSE-spanien Example of the Error message [edited] $./code --2009-04-14 19:01:32-- http://www...

How to read and process binary (base-2) logical representations from file

I have a file containing 800 lines like: id binary-coded-info --------------------------- 4657 001001101 4789 110111111 etc. where each 0 or 1 stands for the presence of some feature. I want to read this file and do several bitwise logical operations on the binary-coded-info (the operations depend on user input and on in...

How can I use paste and awk inside Perl?

I have the following code that uses 'paste' and AWK script inside Perl. use strict; use Data::Dumper; use Carp; use File::Basename; my @files = glob("result/*-*.txt"); my $tocheck = $ARGV[0] || "M"; foreach my $file ( @files ) { my $base = basename($file,".txt"); my @res = `paste <\(awk '\$4 == "M...

How do you use Binary conversion in Python/Bash/AWK?

I am new in binary conversion. I use Python, Bash and AWK daily. I would like to see binary conversion's applications in these languages. For example, I am interested in problems which you solve by it at your work. Where do you use binary conversion in Python/Bash/AWK? I would like to see examples of codes. ...

Shell: insert a blank/new line two lines above pattern

I have a text file and it requires some formatting. I know that if you want to add a blank line above every line that matches your regexp, you can use: sed '/regexp/{x;p;x;}' But I'd like to add a blank line, not one line above, but two lines above the line which matches my regexp. The pattern I'll be matching is a postal code, in th...

awk or sed: Best way to grab [this text]

I'm trying to parse various info from log files, some of which is placed within square brackets. For example: Tue, 06 Nov 2007 10:04:11 INFO processor:receive: [someuserid], [somemessage] msgtype=[T] What's an elegant way to grab 'someuserid' from these lines, using sed, awk, or other unix utility? ...

How can I make Perl's a2p support gawk?

I have some awk scripts that use gawk from cygwin. Now I need to pass these scripts to colleagues that don't have cygwin installed, but do have Perl. I was hoping I can just use a2p that is included in cygwin, but it fails with errors like the following: Undefined subroutine &main::gensub called at ./t.pl line 18, <> line 1. I am hopi...

Anyone know how to make a self-contained Awk/Gawk program on Windows

I'm using an awk script to do some reasonably heavy parsing that could be useful to repeat in the future but I'm not sure if my unix-unfriendly co-workers will be willing to install awk/gawk in order to do the parsing. Is there a way to create a self-contained executable from my script? ...

Unable to make a factorial function in AWK

The code #!/usr/bin/awk # Sed and AWK by O'Reilly (p.179) # Example of what should happen: factorial 5 gives # factorial # Enter number: 3 # The factorial of 3 is 6 BEGIN { printf("Enter number: ") } $1 ~ /^[0-9]+$/ { # assign value of $1 to number & fact number = $1 if (number == 0) ...

Remove lines from file

Hey Guys, I am doing some text processing on a unix system. I have access to the command line on this machine and it has Python, Perl and the default text processing progams installed, awk etc. I have a text file that looks like below: 2029754527851451717 2029754527851451717 2029754527851451717 2029754527851451717 202975452785145...

Unable to combine words in Files by Zsh/AWK

I have File1 A,B,C and File2 D,E,F I am trying to have AD, AE, AF, BD, BE, BF, CD, CE, CF unsuccessfully by echo {`cat File1`}{`cat File2`} giving {A,B,C}{D,E,F} How can you solve the problem by Zsh/AWK? ...

How can I collate and summarize records from a file with Perl?

Hello all, I have a text file in the following format: 211B1 CUSTOMER|UPDATE| 211B2 CUSTOMER|UPDATE| 211B3 CUSTOMER|UPDATE| 211B4 CUSTOMER|UPDATE| 211B5 CUSTOMER|UPDATE| 567FR CUSTOMER|DELETE| 647GI CUSTOMER|DELETE| I want a script that processes the text file and reports the following: "UPDATE" for column CUSTOMER found...

awk: Either modify or append a line, based on its existence

I have a small awk script that does some in-place file modifications (to a Java .properties file, to give you an idea). This is part of a deployment script affecting a bunch of users. I want to be able to set defaults, leaving the rest of the file at the user's preferences. This means appending a configuration line if it is missing, mod...

sed optimization question (large file modification based on smaller dataset)

Hi all, I do have to deal with very large plain text files (over 10 gigabytes, yeah I know it depends what we should call large), with very long lines. My most recent task involves some line editing based on data from another file. The data file (which should be modified) contains 1500000 lines, each of them are e.g. 800 chars long. ...

IP address and Country on same line with AWK

I'm looking for a one-liner that based on a list of IPs will append the country from where the IP is based So if I have this as and input: 87.229.123.33 98.12.33.46 192.34.55.123 I'd like to produce this: 87.229.123.33 - GB 98.12.33.46 - DE 192.34.55.123 - US I've already got a script that returns the country for IP but I need to ...

AWK syntax problem

I want to subtract a column from 100 using awk. I have tried awk '{ t = 100-$2 } END { print t }' /alps/average.log It gave me only the last value subtracted. How it can be accomplished if I want the whole column outputted on terminal? ...

Enumerate substitutions with sed or awk

Given the plain text file with lines bli foo bla abc dfg bli foo bla hik lmn what sed or awk magic transforms it to bli foo_01 bla abc dfg bli foo_02 bla hik lmn so that every occurence of 'foo' is replaced by 'foo_[occurence number]'. ...

How can I mine an XML document with awk, Perl, or Python?

I have a XML file with the following data format: <net NetName="abc" attr1="123" attr2="234" attr3="345".../> <net NetName="cde" attr1="456" attr2="567" attr3="678".../> .... Can anyone tell me how could I data mine the XML file using an awk one-liner? For example, I would like to know attr3 of abc. It will return 345 to me. ...