sed

remove duplicates per line

I have several csv's that look like this: I have several large text files (csv's) that on some lines have redundant entries. That is, due to the way they were merged a certain field will often have the same value twice or three times. It's not always in the same order though. BWTL, NEWSLETTER, NEWSLETTER BWTL, NEWSLETTER, R2R, NEWSLETT...

sed, environment variable and date problem

I want to add a timestamp to server events and store the result in a log. My first idea was : ( ./runServer.sh ) | sed "s/.*/`date +%s` & /" | xargs -0 >Server.log 2>&1 & But it seems sed never reevaluates the date, so all events get the same timestamp. Now I'm trying to get around that using environment variable but I can't find ...

Removing Parts of String That Contain Digit With Sed/Perl (one-liner)

I have a data that looks like this: AB208804_1 446 576 AB208804_1orf 0 AB208804_20 446 576 AB208804_20orf 0 I want to convert them into this: AB208804 446 576 AB208804orf 0 AB208804 446 576 AB208804orf 0 just by removing _\digit part in column 1 and 4. Why this line doesn't work: sed 's/_\d+//g' What's the correct way to do it...

need help! sed or awk how to replace text

i have a json file, some data like this: {"last_modified": {"type": "/type/datetime", "value": "2008-04-01T03:28:50.625462"}, "type": {"key": "/type/author"}, "name": "National Research Council. Committee on the Scientific and Technologic Base of Puerto Rico"s Economy.", "key": "/authors/OL2108538A", "revision": 1} the name's...

How do I replace an arbitrary number of backreferences in sed or Perl? (for obfuscating mailto)

I'm looking for a way to obfuscate mailtos in the source code of a web site. I'd like to go from this: href="mailto:[email protected]" To this: href="" onmouseover="this.href='mai'+'lto:'+'pre'+'sid'+'ent'+'@wh'+'ite'+'hou'+'se.'+'gov'"</code> I'm probably going to go with a PHP solution instead, like this (that way I only h...

sed + print lines in file from the match line until EOF

hi I write the following awk to print lines from the match line until EOF awk '/match_line/,/*/' file how to do the same in sed? Lidia ...

Stripping hex bytes with sed - no match

I have a text file with two non-ascii bytes (0xFF and 0xFE): ??58832520.3,ABC 348384,DEF The hex for this file is: FF FE 35 38 38 33 32 35 32 30 2E 33 2C 41 42 43 0A 33 34 38 33 38 34 2C 44 45 46 It's coincidental that FF and FE happen to be the leading bytes (they exist throughout my file, although seemingly always at the beginnin...

LANG and sed on OSX

In a recent question it was noted that on OSX running sed on a non ascii file gave strange results. For instance if you do (/usr/bin/cal is a random binary file) sed 's/[^A-Z]//' /usr/bin/cal sed will remove all of the printable characters other than A-Z, but many nonprintable characters remain. If however, you do LANG='' sed 's/[...

replace dot with comma in number format

hi... how to change number format from XXXXXX.XXX to XXXXXX,XXX using sed thaks ...

sed or perl + inserting text between two lines

hi all I have the following file: (example.txt showing down) I need to edit the file , the main issue is to append text between two known lines in the file first_line=")" second_line="NIC Hr_Nic (" For example Need to add the following: haattr -add RVG StorageRVG -string haattr -add RVG StorageDG -string haattr -add...

How to go from a multiple line sed command in command line to single line in script

So I have sed running with the following argument fine if I copy and paste this into an open shell: cat test.txt | sed '/[,0-9]\{0,\}[0-9]\{1,\}[acd][0-9]\{1,\}[,0-9]\{0,\}/{N s/[,0-9]\{0,\}[0-9]\{1,\}[acd][0-9]\{1,\}[,0-9]\{0,\}\n\-\-\-//}' The problem is that when I try to move this into a korn shell script, the korn shell throws er...

extract strings out of plist with bash script, maybe sed?

So here is my problem. How can I extract those strings into 3 variables with a bash script? So out of this: <key>GameDir</key> <string>C:/Program Files/</string> <key>GameEXE</key> <string>C:/Program Files/any.exe</string> <key>GameFlags</key> <string>-anyflag</string> I want: GameDir=C:/Program Files/ GameEXE=C:/Program Files/any.e...

Get all occurences of the search pattern

I try to get all occurences of a pattern from a file, but currently I fail if there is more than one occurence per line. Sample line in file: lorem ipsum foo="match1" lorem ipsum foo="match2" lorem ipsum The output I want: match1 match2 I tried getting this using sed: sed -ne 's/^.*foo="\([^"]*\)".*$/\1/p' With this expression ...

Use sed (or other command line) to replace a set of characters with a different set

I need to invert a DNA text string by changing all the A's to T's, T's to A's, C->G, and G->C. Can I elegantly handle this in sed (or other command line) without a whole chain of sed global replace commands? ...

extract sub expression with sed

Hi, I have a list of directories starting from /root. Eg random text /root/dir1/files random end delete me /root/dir1/files/2dir I am waste /root/examples/source alalalala /root/example/header some other text I want to use sed to take from every line the directory and remove everything else. My final output must be /root/dir1/f...

sed output stagnate

I have file names that map to directores. For example. test ---> /to/path/test/program.c I have a line that formats the output of sed into this currently test0 test1 test3 All unique directories, I now need to add leading path and copy their respective c files. Is there a way to stagnate the output of sed while i carry about ther...

unix commandline for inline replacement of all newlines in file with <br>\n

sed 's/$/<br>/' mytext.txt worked but output it all on the command line. I want it to just do the replacement WITHIN the specified file itself. Should I be using another tool? ...

Two variables from a line

The corrupt table entry is logged in the error file something like this... /usr/local/mysql/bin/mysqld: Incorrect key file for table './accounts/headers.MYI'; try to repair it I want to write a query that will repair this table. REPAIR TABLE accounts.headers; What I need to do is search for the first instance of "./" an...

What's the meaning of this sed command? sed 's%^.*/%%'

I saw a bash command sed 's%^.*/%%' Usually the common syntax for sed is sed 's/pattern/str/g', but in this one it used s%^.* for the s part in the 's/pattern/str/g'. My questions: What does s%^.* mean? What's meaning of %% in the second part of sed 's%^.*/%%'? ...

Words before and after selected text

My SQL query looks like this... sekect * FROM mail_headers a LEFT JOIN mail_headers_body b ON a.mailid=b.id blah blah blah I can select the lines before and after the result, using -A, -B switches. But how do I find the 4 words before and after the selected text? grep -i 'JOIN mail_headers_body' mysql-gen.log This will retur...