awk

Awk, Sed : How to parse and sum values from a string

Hi all, I've a string like this Cpu(s): 1.9%us, 2.1%sy, 1.5%ni, 94.5%id, 0.8%wa, 0.0%hi, 0.1%si, 0.0%st it represents the Cpu usage of my unix box. Now I need to apply awk and sed (i think) to extract the current load of my CPUs. I'd like to extract the 'us', 'sy', 'ni' values from the string and then I want to sum them. The s...

compare time in awk

Hi, I want to check the modified time of all the files in one directory. If any file has the time different from the current system time, then RED will be printed. Otherwise , GREEN is printed. ls -lrt| grep main | awk '{if ($8 != date +%R ) print "-->RED"}END{print "-->GREEN"}' May you suggest me how to correct the above sta...

mac unix script problem

I'm trying to write a script that breaks up a VERY large file into smaller pieces that are then sent to a script that runs in the background. The motivation is that if the script is running in the background, I can run in parallel. Here is my code, ./seq works just like the normal seq command (which mac doesn't have). and $1 is the huge...

Regarding UNIX Grep Command

I need to write a shell script that pick all the files (not directories) in /exp/files directory. For each file inside the directory I want to find whether the last line of file is received . The last line in the file is a trailer record. Also the third field in the last line is the number of data records count i.e 2315 (Total Numbe...

how to remove leading whitespace from each line in a file?

I have a file that looks something like this: for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) I want it to look like this (remove indentations): for (i = 0; i < 100; i++) f...

sorting group of lines

I have a text file like below iv_destination_code_10 TAP310_mapping_RATERUSG_iv_destination_code_10 RATERUSG.iv_destination_code_10 = WORK.maf_feature_info[53,6] iv_destination_code_2 TAP310_mapping_RATERUSG_iv_destination_code_2 RATERUSG.iv_destination_code_2 = WORK.maf_feature_info[1,6] iv_destination_code_3 TAP310_mapping_RATERUSG_iv...

How can I pull a line out of /etc/passwd if its corresponding shadow entry is 999999?

I want to compare each user in the passwd file with his entry in the shadow file, and print out the whole line of the passwd file if the entry in the shadow file matches 999999. What is the easiest way in Perl to do this? Or I suppose I could awk the values out of one file and match in the other file? What is the best way of doing this...

Shell Script, Search File for String

I'm writing a shell script that opens a file and needs to find a tag like ##FIND_ME##. The string I'm searching for is a constant (and there is only ever one instance of it.) Once I locate that string, I need it to start a new search for a different string from that point forward. My *nix skills are a little rusty, should try to imple...

Awk to replace single quote

Hi I am stuck with this I want to replace all include('./ in a set of files with include('. I am trying to use awk as follows: awk '{gsub("include\('"'"'./", "include\('"'"'", $0); print > FILENAME}' *.php It throws me this error. awk: (FILENAME=xyz.php FNR=1) fatal: Unmatched ( or \(: /include('.// Any help would be appreciated....

How to extract the name of immediate directory along with the filename?

I have a file whose complete path is like /a/b/c/d/filename.txt If I do a basename on it, I get filename.txt. But this filename is not too unique. So, it would be better if I could extract the filename as d_filename.txt i.e. {immediate directory}_{basename result} How can I achieve this result? ...

How to remove empty tables from a MySQL backup file.

I have multiple large MySQL backup files all from different DBs and having different schemas. I want to load the backups into our EDW but I don't want to load the empty tables. Right now I'm cutting out the empty tables using AWK on the backup files, but I'm wondering if there's a better way to do this. If anyone is interested, this i...

Joining Line Breaks in FASTA file With Condition in SED/AWK/Perl one-liner

I have a data that looks like this > sq1 foofoofoobar foofoofoo > sq2 quxquxquxbar quxquxquxbar quxx > sq3 paxpaxpax pax What I want to do is to join them into one lines: > sq1 foofoofoobarfoofoofoo > sq2 quxquxquxbarquxquxquxbarquxx > sq3 paxpaxpaxpax I tried this code but fail. sed -e 'te' -e 'H;$!d;:e' -e 'x;/^$/d;s/\n//g' Wh...

Using sed or awk to update a line

I want to open the httpd.conf file and change the LogFormat line with the new parameters. The criterion will be that the line should start with "LogFormat" and end with the word "combined" Here is how I do manually. I want to change the line programatically. vi /etc/httpd/conf/httpd.conf #LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Refer...

Sed script to edit csv file Or Python

In our project we need to import the csv file to postgres. There are multiple types of files meaning the length of the file changes as some files are with fewer columns and some with all of them. We need a fast way to import this file to postgres. I want to use COPY FROM of the postgres since the speed requirement of the processing are ...

sed or awk multiline replace

Hi guys, I am trying to append formatting to all /* TODO : ... */ tags, but I am having trouble in the multi-line area. I can do single line sed's; but for multiline sed and awk, I don't know. How do I do this? I'm open to either. Here's what I have so far. sed 's/\/\/\*[ \t]*TODO[ \t]*:.*/*\//<span style="color:#aaaaaa;font-weight:bo...

Solaris awk Troubles

I'm writing a shell script and I need to strip FIND ME out of something like this: * *[**FIND ME**](find me)*&#x0d; and assign it to an array. I had the code working flawlessly .. until I moved the script in Solaris to a non-global zone. Here is the code I used before: objectArray[$i]=`echo $line | nawk -F '*[**|**]' '{print $2}'` ...

gawk / awk: piping date to getline *sometimes* won't work

Hi! I'm attempting to convert dates from one format to another: From e.g. "October 29, 2005" to 2005-10-29. I have a list of 625 dates. I use Awk. The conversion works -- most of the time. Hovewer, sometimes the conversion won't happen at all, and the variable supposed to hold the (converted) date remains undefined. This always happen...

Calculate difference of numbers in two files

Say I have two files where there is one number per line File 1 file 2 0.12 0.11 0.121 0.454 .... .... I want to create file or output difference between each number on to the screen, so that result looks like 0.0099 -0.333 ...... You can use bash/awk/sed ...

awk or perl one-liner to print line if second field is longer than 7 chars.

I have a file of 1000 lines, each line has 2 words, seperated by a space. How can I print each line only if the last word length is greater than 7 chars? Can I use awk RLENGTH? is there an easy way in perl? ...

problem with awk script

Hello, when I call my awk script, I keep getting an error : sam@sam-laptop:~/shell/td4$ awk -f agenda.awk -- -n Robert agenda.txt awk: agenda.awk:6: printf "Hello" awk: agenda.awk:6: ^ syntax error the script contains this : #!/usr/bin/awk BEGIN { } printf "Hello" END { } Thank you ...