awk

Delete strings from an html file containing a pattern using unix commands

Hi, I have a messy html that looks like this: <div id=":0.page.0" class="page-element" style="width: 1620px;"> <div> <img src="viewer_files/viewer_004.png" class="page-image" style="width: 800px; height: 1131px; display: none;"> <img src="viewer_files/viewer_005.png" class="page-image" style="width: 1600px;"> </div> </div>// this ...

scanf,back reference in awk

Is there any implementation of scanf()(like in C) in awk(POSIX)? I know that awk does not have back reference like in sed and perl. What is the easiest way to simulate in awk? Thanks Nyan ...

Remove non-ascii characters from csv

I want to remove all the non-ASCII characters from a file in place. I found one solution with tr, but i guess I need to write back that file after modification. I need to do it in place with relatively good performance. Any suggestions? Thanks ...

How to get the PID of a process in a pipeline

...

Convert file one word per line

I would like to convert a text file containing words, space separate to a file where each word appears on a separate line sdf sdfsd= sdfsdf sdfsdf will become sdf sdfsd= sdfsdf sdfsdf thanks ...

Does an AWK script take up a lot of CPU?

Does AWK uses a lot of processing power? If so, is there a better scripting language to do it? Or should I do it in C itself (where rest of my code is). ...

parsing multiple values from a file

Hi I have a file that is just one line (one HUGE line) to parse. I want to parse out the value that appears between "Undefined error code" and " id" on this line. The thing is this appears multiple times on the same line with different values everywhere. The following code only gives me the last instance. cat bad_events_P2J3.xml | sed...

Differences/merging two files

Hey! I have two lists of IP addresses. I need to merge them into three files, the intersection, those from list1 only and those from list2 only. can I do this with awk/diff or any other simple unix command? How? The files look like this: 111.222.333.444 111.222.333.445 111.222.333.448 Thank you! ...

awk-1-liner to sum numbers in stdin, while printing number of occurrences of certain text along with other arbritrary text

I'm definitely interested in getting a book on awk. I've been won over, despite playing with it for little time. However, in the mean time, I have a problem I suspect I could solve exclusively with [g]awk. To demonstrate, I'll use some fdisk output. In this example, the desired end result would be something like: Disks: 2 [240 GB total]...

awk optimization

any pointers on what must not be in an awk script? I am asking as I dont find any tool to debug an awk script. I have a script taking a lot of cpu so need to know if I am doing something terribly wrong with the script. Just for an example, I keep looking for output of a logfile via 'tail -f filename' till my script gets killed. ...

How to find the biggest element and store it in array

I have file as shown below .I have to find the maximum value for each timestamp. So I have to find the max(2434, 2681, 2946, 1626) , max(3217, 4764, 4501, 3372) and so on ... (since these numbers have a common timestamp) Timestamp value 1280449531 2434 1280449531 2681 1280449531 2946 1280449531 1626 1280449532 3217 1280449532 4764 128...

how to subtract a constant number from a column

Is there a another way to subtract the smallest value from all the values of a column, effectively offset the values? I have to subtract the first number in teh 1st column from all other numbers in teh first column.. I wrote this script which is not giving the right result . can anybody help me ? awk '{$1 = $1 - 1280449530}' file ...

How to reverse order of fields using AWK?

I have a file with the following layout: 123,01-08-2006 124,01-09-2007 125,01-10-2009 126,01-12-2010 How can I convert it into the following by using AWK? 123,2006-08-01 124,2007-09-01 125,2009-10-01 126,2009-12-01 ...

Wrap Field in a Line with Variable Number of Separators

Hello all, I have a file that uses the pipe character as a separator since I assumed that most of my clients wouldn't use the pipe character in their data. Apparently I was wrong, but I compensated for this by specifying that any fields using the pipe character need to be wrapped in double quotes. Unfortunately, they haven't done this, ...

Find and Copy a String within HTML code

I'm trying something new, I would normally do this in C# or VB. But for speed reason I'd like to do this on my server. Open File terms.txt Take each item one at a time from terms.txt and open a url (possibly curl or something else) and go to http://system.com/set=terms View the HTML source and extract pic names (stringB). Look for ima...

Copy a section into a new file

I have a text file, !--- Marker one --! aaaa bbbb cccc dddd !--- Marker two --! eeee ffff !--- Marker three --! gggg hhhh iiii And I need to copy from Marker two (till the end of Marker two) into a new file, using Bash. !--- Marker two --! eeee ffff Becomes a separate file. ...

Filtering Rows Based On Number of Columns with AWK

I have lines of data that contain single column and two columns. What I want to do is to extract lines that contain only 2 columns. 0333 foo bar 23243 qux yielding only: 0333 foo 23243 qux Note that they are tab separated, even for lines with only one column you have tab at the beginning. What's the way to do it? I tried this...

In a shell (bash) How can I execute more than one command in a pipeline?

I'm trying pipe the output of an awk command through more than one command at once in a Bash shell, following my knowledge I'm coming up with this: awk '$13 ~ /type/ {print $15}' filename.txt | (wc -l || sort -u) I want the result of the awk command to be both counted AND sorted, how can I accomplish that? Even with && command it does...

How can I delete one line before and two after a string?

Say you have records in a text file which look like this: header data1 data2 data3 I would like to delete the whole record if data1 is a given string. I presume this needs awk which I do not know. ...

need help! sed or awk how to replace text

i have a json file, some data like this: {"last_modified": {"type": "/type/datetime", "value": "2008-04-01T03:28:50.625462"}, "type": {"key": "/type/author"}, "name": "National Research Council. Committee on the Scientific and Technologic Base of Puerto Rico"s Economy.", "key": "/authors/OL2108538A", "revision": 1} the name's...