Using sed to replace a string with the contents of a variable, even if it's an escape character.

Question

I'm using sed -e "s/\*DIVIDER\*/$DIVIDER/g" to replace *DIVIDER* with a user-specified string, which is stored in $DIVIDER. The problem is that I want them to be able to specify escape characters as their divider, like \n or \t. When I try this, I just end up with the letter n or t, or so on.

Does anyone have any ideas on how to do this? It will be greatly appreciated!

EDIT: Here's the meat of the script, I must be missing something.

curl --silent "$URL" > tweets.txt

if [[ `cat tweets.txt` == *\<error\>* ]]; then
    grep -E '(error>)' tweets.txt | \
    sed -e 's/<error>//' -e 's/<\/error>//' |
    sed -e 's/<[^>]*>//g' |

head $headarg | sed G | fmt

else
    echo $REPLACE | awk '{gsub(".", "\\\\&");print}'
    grep -E '(description>)' tweets.txt | \
    sed -n '2,$p' | \
    sed -e 's/<description>//' -e 's/<\/description>//' |
    sed -e 's/<[^>]*>//g' |
    sed -e 's/\&amp\;/\&/g' |
    sed -e 's/\&lt\;/\</g' |
    sed -e 's/\&gt\;/\>/g' |
    sed -e 's/\&quot\;/\"/g' |
    sed -e 's/\&....\;/\?/g' |
    sed -e 's/\&.....\;/\?/g' |
    sed -e 's/^  *//g' |
    sed -e :a -e '$!N;s/\n/\*DIVIDER\*/;ta' |   # Replace newlines with *divider*.
    sed -e "s/\*DIVIDER\*/${DIVIDER//\\/\\\\}/g" |          # Replace *DIVIDER* with the actual divider.

    head $headarg | sed G
fi

The long list of sed lines are replacing characters from an XML source, and the last two are the ones that are supposed to replace the newlines with the specified character. I know it seems redundant to replace a newline with another newline, but it was the easiest way I could come up with to let them pick their own divider. The divider replacement works great with normal characters.

Answer 1

You just need to escape the escape char.

\n will match \n

\ will match \

\\ will match \

Answer 2

You can use bash to escape the backslash like this:

sed -e "s/\*DIVIDER\*/${DIVIDER//\\/\\\\}/g"

The syntax is ${name/pattern/string}. If pattern begins with /, every occurence of pattern in name is replaced by string. Otherwise only the first occurence is replaced.

Answer 3

Maybe:

case "$DIVIDER" in
(*\\*) DIVIDER=$(echo "$DIVIDER" | sed 's/\\/\\\\/g');;
esac

I played with this script:

for DIVIDER in 'xx\n' 'xxx\\ddd' "xxx"
do
    echo "In:  <<$DIVIDER>>"
    case "$DIVIDER" in     (*\\*) DIVIDER=$(echo "$DIVIDER" | sed 's/\\/\\\\/g');;
    esac
    echo "Out: <<$DIVIDER>>"
done

Run with 'ksh' or 'bash' (but not 'sh') on MacOS X:

In:  <<xx\n>>
Out: <<xx\\n>>
In:  <<xxx\\ddd>>
Out: <<xxx\\\\ddd>>
In:  <<xxx>>
Out: <<xxx>>

Answer 4

It seems to be a simple substitution:

$ d='\n'
$ echo "a*DIVIDER*b" | sed "s/\*DIVIDER\*/$d/"
a
b

Maybe I don't understand what you're trying to accomplish.

Then maybe this step could take the place of the last two of yours:

sed -n ":a;$ {s/\n/$DIVIDER/g;p;b};N;ba"

Note the space after the dollar sign. It prevents the shell from interpreting "${s..." as a variable name.

And as ghostdog74 suggested, you have way too many calls to sed. You may be able to change a lot of the pipe characters to backslashes (line continuation) and delete "sed" from all but the first one (leave the "-e" everywhere). (untested)

Answer 5

Using FreeBSD sed (e.g. on Mac OS X) you have to preprocess the $DIVIDER user input:

d='\n'
d='\t'
NL=$'\\\n'
TAB=$'\\\t'
d="${d/\\n/${NL}}"
d="${d/\\t/${TAB}}"
echo "a*DIVIDER*b" | sed -E -e "s/\*DIVIDER\*/${d}/"