awk

calculate the difference from flat file

I have a text file and the last 2 lines look like this... Uptime: 822832 Threads: 32 Questions: 13591705 Slow queries: 722 Opens: 81551 Flush tables: 59 Open tables: 64 Queries per second avg: 16.518 Uptime: 822893 Threads: 31 Questions: 13592768 Slow queries: 732 Opens: 81551 Flush tables: 59 Open tables: 64 Queries per s...

Invoke another command line app from within an awk block

I want to invoke curl (which invokes a REST service) for every matching row processed via awk. Input file (delimited by colon). e.g. $ cat /tmp/input tiger:EN haiti:FR federer:CH I got as far as this: awk -F':' '{print $1 "=" $2}' /tmp/input This just confirms that I can extract the columns out correctly. Now I need to invoke th...

rearrange column's elements and display count

Hi , i have text file like this: i am fine how are you what i how are i need an output like below: i : 2 am : 1 fine : 1 how : 2 are : 2 you : 1 what : 1 there can be many repititions of the words: how could i do this using a shell script or an awk? ...

awk won't print new line characters

I am using the below code to change an existing awk script so that I can add more and more cases with a simple command. echo `awk '{if(/#append1/){print "pref'"$1"'=0\n" $0 "\n"} else{print $0 "\n"}}' tf.a note that the first print is "pref'"$1"'=0\n" so it is referring to the variable $1 in its environment, not in awk itself. The co...

editing text files with perl

I'm trying to edit a text file that looks like this: TYPE=Ethernet HWADDR=00:.... IPV6INIT=no MTU=1500 IPADDR=192.168.2.247 ... (Its actually the /etc/sysconfig/network-scripts/ifcfg- file on red hat Linux) Instead of reading and rewriting the file each time I want to modify it, I figured I could use grep, sed, awk or the native text ...

How can I reformat the GECOS field with Perl or awk?

I want to scan the passwd file and change the order of words in the comment field from firstname lastname to lastname firstname, and force the surname to capitals. So, change every line from: jbloggs:x:9999:99:Joe Bloggs:/home/jbloggs:/bin/ksh to: jbloggs:x:9999:99:BLOGGS Joe:/home/jbloggs:/bin/ksh I'm new to Perl and I'm having p...

SVG to black-and-white

Hi, I would like to be able to convert SVG documents to black and white. My try is the following Makefile script using 'sed' : %.bw.svg: %.svg sed '/stroke:none/!s/stroke:[^;\"]*/stroke:black/g' $< > $@ This works for lines etc but not for fillings. Basically if the stroke is not invisible (none), then I convert it to black. I wou...

Using awk (or sed) to remove newlines based on first character of next line

Hello everyone, here's my situation: I had a big text file that I wanted to pull certain information from. I used sed to pull all the relevant information based on regexp's, but each "piece" of information I pulled is on a separate line, I'd like for each "record" to be on its own line so it can be easily imported into a DB. Here's a sam...

How to use getline in AWK with a command that is concatenated together

This is driving me insane. All I want to do is pass a command to the terminal from awk, where the command is a string concatenated together made from other variables. The documentation for awk says that something like "echo" $1 | getline var should put the value of $1 into var. But this is not the case. What am I missing here? I sh...

how do I paste text to a line by line text filter like awk, without having stdin echo to the screen?

I have a text in an email on a windows box that looks something like this: 100 some random text 101 some more random text 102 lots of random text, all different 103 lots of random text, all the same I want to extract the numbers, i.e. the first word on each line. I've got a terminal running bash open on my Linux box... If these were ...

With sed or awk, how do I match from the end of the current line back to a specified character?

I have a list of file locations in a text file. For example: /var/lib/mlocate /var/lib/dpkg/info/mlocate.conffiles /var/lib/dpkg/info/mlocate.list /var/lib/dpkg/info/mlocate.md5sums /var/lib/dpkg/info/mlocate.postinst /var/lib/dpkg/info/mlocate.postrm /var/lib/dpkg/info/mlocate.prerm What I want to do is use sed or awk to read f...

Concatenating multiple lines with a discriminator

I have the input like this Input: a,b,c d,e,f g,h,i k,l,m n,o,p q,r,s I wan to be able to concatenate the lines with a discriminator like "|" Output: a,b,c|d,e,f|g,h,i k,l,m|n,o.p|q,r,s The file has 1million lines and I want to be able to concatenate lines like the example before. Any ideas about how to approach this? ...

Extraction of data from a simple XML file

I've a XML file with the contents: <?xml version="1.0" encoding="utf-8"?> <job xmlns="http://www.sample.com/"&gt;programming&lt;/job&gt; I need a way to extract what is in the <job..> </job> tags, programmin in this case. This should be done on linux command prompt, using grep/sed/awk. ...

What's the clearest way to replace trailing backslash \ with \n?

I want multi-line strings in java, so I seek a simple preprocessor to convert C-style multi-lines into single lines with a literal '\n'. Before: System.out.println("convert trailing backslashes\ this is on another line\ \ \ above are two blank lines\ But don't convert non-trailing backslashes, like: \"\t\" and \'\\\'"); After: ...

Passing variable to awk and using that in a regular expression

I'm learning awk and I have trouble passing a variable to the script AND using it as part of a regex search pattern. The example is contrived but shows my probem. My data is the following: Eddy Smith 0600000000 1981-07-16 Los Angeles Frank Smith 0611111111 1947-04-29 Chicago Victoria McSm...

Emacs: Is there a way to create a interactive script using Emacs?

I am new to emacs, but shocked at what I can really do and how much time it saves (Macros save A LOT of time). But I was wondering it was possible to create step based scripts where it asks the user for input and executes code based on that. For example maybe I want to create a SQL query so it would prompt something like: >table name? m...

How to remove the filename from the end of the du -h output

Hi, take the example: $ du -h file_size.txt 112K file_size.txt How would i remove the filename from the output of du -h I have tried to use sed to search for a string (the filename) and replace it with nothing, but it hasnt worked (command below) du -h file_size.txt | sed 's/ 'file_size.txt'//' Could someone please point out why...

counting non blank lines per block of a file, having several blocks separated by a blank line

Hi, I'd like to count lines in a file that consists of several blocks, say 3, each with a different number of lines. Each block is separated by a blank line. Is there a one line solution? So far here is what I have: awk '(NR>4) && NF!=0 {++count} END {print count}' filename > outfile This obviously counts all non-blank lines (and get...

processing text from a non-flat file (to extract information as if it *were* a flat file)

Hi, I have a longitudinal data set generated by a computer simulation that can be represented by the following tables ('var' are variables): time subject var1 var2 var3 t1 subjectA ... t2 subjectB ... and subject name subjectA nameA subjectB nameB However, the file generated writes a data file in a format similar to the f...

Any alternatives to this awk script to soothen subversion pain ?

On my development server, I run svn updates to deploy bug fixes or changes to the webapp's code. Normally I run: svn stat --show-updates and then selectively chose which files to update; appending the selected files to the end of a svn update command. I miss GIT's command line interface and as a concession, I just want to improve the...