awk

problem in shell script

I am trying this option #!/bin/ksh echo $1 awk '{FS="=";print $2}' $1 and on the command line test_sh INSTANCE=VIJAY but awk is failing. Is there any problem here? Basically I need the value VIJAY passed on the command line. ...

Get only a part of the line using awk

I have a log file and i use awk to extract the required columns. My command is awk '{ print $2" "$5" " $6}' log.txt The $5 column contains the servername and the format can be like @:server1@:@:@:, server2@:@:@:@:@:, @:@:Server3 with no fixed amount of @: symbols. How do I modify my statement so that I remove all the '@:' from the c...

convert exponential to decimal in python

I have an array in python that contains a set of values, some of them are 2.32313e+07 2.1155e+07 1.923e+07 11856 112.32 How do I convert the exponential formats to the decimal format Additional: Is there a way I can convert the exponent directly to decimal when printing out in UNIX with awk? ...

How can I split a file into a given number of parts in Perl? (On Unix)

I have several large files, each of which I want to chunk/split it in to predefined number of parts. Is there an efficient way to do it in Unix (e.g. via awk/sed/perl)? Also each file can have varied number of lines. File1.txt 20,300,055 lines File2.txt 10,033,221 lines etc... ...

Awk/Sed: How to do a recursive find/replace of a string?

How to I find and replace every occurrence of: subdomainA.example.com with subdomainB.example.com in every text file under the /home/www/ directory tree (recursive find/replace). ...

Output on a single line

The following code is working as expected. But I can not format the output. It will print something like this: mysql test someDB I want the output on a single line mysql test someDB I tried using sed in the script but it did not work. #!/bin/sh for dbName in `mysqlshow -uroot -pPassWord | awk '{print $2}'` do echo "$dbName" | egr...

How to delete partial duplicate lines with AWK?

I have files with these kind of duplicate lines, where only the last field is different: OST,0202000070,01-AUG-09,002735,6,0,0202000068,4520688,-1,0,0,0,0,0,55 ONE,0208076826,01-AUG-09,002332,316,3481.055935,0204330827,29150,200,0,0,0,0,0,5 ONE,0208076826,01-AUG-09,002332,316,3481.055935,0204330827,29150,200,0,0,0,0,0,55 OST,0202000068,...

Remove strings after each first word in a text file

File1: hello (OPTION1) 123456 123456 123456 world (OPTION1) 123456 123456 123456 foo (OPTION1) 123456 123456 123456 bar (OPTION1) 123456 123456 123456 How would one remove each string after each first word in the textfile File1? This would probably be down with awk/sed/cat - but I cannot figure it...

Move line(s) to follow another line in a file

I got a file that has a line in the file like this: check=('78905905f5a4ed82160c327f3fd34cba') I'd like to be able to move this line to follow a line that looks like this: files=('somefile.txt') The array though at times that can span multiple lines, for example: files=('somefile.txt' 'file2.png' 'another.txt' ...

Simplest method to get the dictionary definitions of a list of words in a text file

File1: hello world I don't know the best method to extract a list of words from a text file, find their definitions and paste them into an output textfile. I've been thinking of using WordNet - but don't know how to automate the process. Does anyone have any ideas (perhaps google/APIs/linux applications) that one could use to find th...

Print 3rd column to last column in Awk

I'm trying to parse out a DbgView log file the first three columns of which I'm not interested in. I can't seem to find an example that prints from column 3 onwards until the end of the line. Note that each line has variable number of columns. ...

Shell: simple way to get all lines before first blank line

What's the best shell command to output the lines of a file until you encounter the first blank line? For example: output these lines but do not output anything after the above blank line (or the blank line itself) awk? something else? ...

create an xml file using a shell script

I have a table with two columns column_1 column_1 12345 12345 73255 73255 71377 71377 Now i want to create an xml like <header> <value>12345</value> <value>73255</value> <value>71377</value> <footer> basically i need to use a select query and put any one of the fields into the values of xml. could you please suggest h...

What is wrong with this AWK script?

Basically, I am creating an XML file by taking the values from a column of a table. I am starting an AWK script from a shell script (ksh if it matters) like this: SQL_RESULT=`sqlplus -s ${CONNECT_STRING} << EOF ${SQLPLUS_SETTINGS} select customer_id from GD9_GENTH_CUST_SUBSCR; exit; EOF` FILE_LIST=`echo $SQL_RESULT|sed -e 's/\n/''/g'` ...

How do I split up a line and rearrange its elements?

I have some data on a single line like below abc edf xyz rfg yeg udh I want to present the data as below abc xyz yeg edf rfg udh so that alternate fields are printed with newline separated. Are there any one liners for this? ...

Send the result of multiple commands to one text file?

This is what I have so far - my dropbox public URL creation script for a directory of public URLs (getdropbox.com - gpl I think). My LIST file was created using ls in the following fashion: ls -d ~/Dropbox/Public/PUBLICFILES/* > LIST dropboxpuburl.sh: for PATH in `cat LIST` do echo $PATH dropbox puburl $PATH > ~/URLLIST/$PATH d...

Remove odd lines in a text file

File: /home/USER/DIR/a http://www.here.is.a.hyper.link.net/ /home/USER/DIR/b http://www.here.is.another.hyper.link.net/ Need to remove all the odd lines in this file (PUBLIC-DIRECTORY-LIST)? Its for my batch script which can be found below (dropbox batch puburl creator): for PATH in `cat LIST` do echo $PATH dropbox puburl $PATH done ...

Replace two or more spaces within a text file with a ;

File1: hello world foo bar a word with a space I need to replace all white spaces which are two or more in length with a semi-colon(;). Result: File2: hello;world foo;bar a;word with a space ...

Display and pipe only a specific line of text from a stream

Here is a command line script for a dictionary lookup using Wordnet: #!/bin/bash # Command line look up using Wordnet - command line dictionary echo "Type in your word:" read word /usr/bin/curl -s -A 'Mozilla/4.0' 'http://wordnetweb.princeton.edu/perl/webwn?s='$word'&amp;sub=Search+WordNet&amp;o2=&amp;o0=1&amp;o7=&amp;o5=&amp;o1=1&am...

Remove a variety of lines in a text file

I've been trying to implement a bash script that reads from wordnet's online database and have been wondering if there is a way to remove a variety text files with one command. Example FileDump: **** Noun **** (n)hello, hullo, hi, howdy, how-do-you-do (an expression of greeting) "every morning they exchanged polite hellos" **** Verb **...