sed

Bash script for downloading files with Curl

I'm trying to knock together a little batch script for downloading files, that takes a URL as its first parameter and a local filename as its second parameter. In testing I've learned that its tripping up on spaces in the output filename, so I've tried using sed to escape them, but its not working. #!/bin/bash clear echo Downloading $1...

Extract Data from CSV in Bash script (Sed, AWK, Grep?)

I need to extract some data from a CSV file. The CSV is a 2 column file with multiple records. The first column is the date, the second column is the data that needs to be extracted. The first row of the CSV file is the column headers, so it can be skipped. And I've already created the column header for the extracted data's csv file,...

sed: Search and replace varying column

Hello everyone. I got a document that fields are separated by a colon(:) where I need to change the second field from time to time. The document looks like this: name1:UhX.PPFW7$YhPMH0BISY:23490:::::: name2:1./0oV$GEs6WJpE$LHXhy:19239:0:29388:2::29302: ... The second field of the file will change occasionally and may contain a coupl...

Is this a valid shell sed command and what does it do?

sed 's/\([ab]\)\([ab]\)./\2\1x/g' file.txt My understanding is: in file.txt, find any string start with either 'a' or 'b', followed by either an 'a' or a 'b', followed by any ONE char (say 'abc'), and replace it with the string 1st, and 2nd char switched palce, and 3rd char is x ('bax'). Is the syntax of the command correct? and am I c...

stdout to variable. with mktmp, sed, grep, etc. line by line

Hi, I use ldapsearch to get some users from my LDAP-Server. The command replies something like this: uid: name.surname homeDirectory: /home/name sambaSID: S-1-4-32-224545876-87201423761-4821562975-6853 sambaHomeDrive: G: description: poI description: pPI sn: naut givenName: givenName: peter mail: [email protected] Now I want to assi...

Joining Line Breaks With Condition in SED

I have data that looks like this: > sq1 foofoofoobar foofoofoo > sq2 quxquxquxbar quxquxquxbar quxx > sq3 foofoofoobar foofoofoo > sq4 foofoofoobar foofoo I want to join the lines on the basis of ">sqi" header as cut-off line, i.e. yielding: foofoofoobarfoofoofoo quxquxquxbarquxquxquxbarquxx foofoofoobarfoofoofoo foofoofoobarfoofoo ...

bash: Using sed in backticks

I want to escape some special chars inside a string automatically. I thought of echoing that string and pipe it through some seds. This doesn't seem to work inside of backticks. So why does echo "foo[bar]" | sed 's/\[/\\[/g' return foo\[bar] but FOO=`echo "foo[bar]" | sed 's/\[/\\[/g'` && echo $FOO just returns foo[bar] ? ...

Replacing specific non-printable characters in huge files from linux command line

I need to replace the ascii characters SOH and STX (start of header and start of text, ascii characters 1 and 2, respectively) in some really huge text files as quickly as possible... Is sed the way to go? What does that command look like? ...

Gnu Makefile - Handling dependencies

What approach do C++ programmers on Unix platform use to create and manage Makefiles? I was using hand made Makefiles for my projects but they don't handle header file changes and other dependencies. I googled around and found a good solution here. But I ran into a problem here in the sed command - sed -e 's/#.*//' -e 's/^[^:]*: ...

Bash - Format a Textfile - Pair every two lines

Hi, With a simple bash script i generate a text file with many entrys like this: 192.168.1.1 hostname1 192.168.1.2 hostname2 192.168.1.3 hostname3 Now i want to reformat this file, that it looks like this: 192.168.1.1 hostname1 192.168.1.2 hostname2 192.168.1.3 hostname3 Some ideas to solve this? Sed maybe? Thanks for help and be...

Grep with "half"-exact match

Hi, I try to use grep to match a case like this: I have a list of hostname and ips in this format: 238-456 192.168.1.1 ak238-456 192.168.1.2 238-456a 192.168.1.3 458-862 192.168.1.4 Now i try to grep "238-456" but of course this returns: 238-456 192.168.1.1 and... ak238-456 192.168.1.2 and... 238-456a 192.168.1.1 but i only...

Single sed command for multiple substitutions?

I use sed to substitute text in files. I want to give sed a file which contains all the strings to be searched and replaced in a given file. It goes over .h and .cpp files. In each file it searches for file names which are included in it. If found, it substitutes for example "a.h" with "<a.h>" (without the quotes). The script is this: ...

Replace \n with \r\n in Unix file

I'm trying to do the opposite of this question, replacing Unix line endings with Windows line endings, so that I can use SQL Server bcp over samba to import the file. I have sed installed but not dos2unix. I tried reversing the examples but to no avail. Here's the command I'm using. sed -e 's/\n/\r\n/g' myfile I executed this and the...

Converting XML input from multiple lines to one line

I have an xml like this <address> <street>abc</street> <number>123</number> </address> <address> <street>abc1</street> <number>345</number> </address> ... ... <address> <street>xyz</street> <number>999</number> </address> I want to be able to convert this to <address><street>abc</street><number>123</number></addr...

Finding a pattern and extracting it into another file using sed in BASH

I have a few SQL scripts which contains table creation SQL statements. I want to remove all the statements having foreign key mappings. For example below is one script. CREATE TABLE x ( ORG_ID NUMBER primary key, BILLING_doc xmltype ); alter table x add (constraint fk_x foreign key (org_id) references organization\r (org...

How can I replace intraline tabs with spaces, maintaining alignment?

I like to use spaces for indentation rather than tabs; replacing tabs at the beginning of a line is easy in sed or vim: s/^I/ /g But if there are tabs within a line (pretend the spaces are the width of the tab char): 'foo'^I ^I => 'bar', 'bazzle'^I => 'qux', Each tab doesn't correspond to a set number of spaces to maintain the...

How can I apply Unix's / Sed's / Perl's transliterate (tr) to only a specific column?

I have program output that looks like this (tab delim): $ ./mycode somefile 0000000000000000000000000000000000 238671 0000000000000000000000000000000001 0 0000000000000000000000000000000002 0 0000000000000000000000000000000003 0 0000000000000000000000000000000010 0 000000000000000000...

Uppercasing First Letter of Words Using SED

How do you replace the first letter of a word into Capital letter, e.g. Trouble me Gold rush brides into Trouble Me Gold Rush Brides ...

Using sed to dynamically generate a file name

I have a CSV file that I'd like to split up based on a field in the file. Essentially, there can be two brands, GVA and HBVL. I'd like to split the file into a file for each brand before I import it into a database. Sample of the CSV file "D509379D5055821451C3695A3752DCCD",'1900-01-01 01:00:00',"M","1740","GVA",'2009-07-01 13:25:00',0 ...

Converting FASTQ to FASTA with SED/AWK

I have a data in that always comes in block of four in the following format (called FASTQ): @SRR018006.2016 GA2:6:1:20:650 length=36 NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNGN +SRR018006.2016 GA2:6:1:20:650 length=36 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!+! @SRR018006.19405469 GA2:6:100:1793:611 length=36 ACCCGCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC +SRR...