Help with variables and new lines, and quoting in a bash script

Question

I would like to automate the following svn command. Note this command produces the desired results on my system - Ubuntu 10.04, svn 1.6.6, bash shell, when issued from the command line:

svn ci -m $'Added new File: newFile.txt\nOrig loc: /etc/networking/newFile.txt' /home/user/svnDir/newFile.txt

I would like to run that command in a bash script, assuming that the original full path to the file is contained in the variable $oFileFull, and the filename is in $oFileName. The script is executed from the svn directory. I need to allow for the possibility that the file name and or path contain spaces.

so the line inside my shel script might look like:

svn ci -m$'Added new file: ${oFileName}\nOrig loc: ${oFileFull}' ${oFileName}

But I want the variables (which may contain spaces) expanded before the command is executed, and I cannot figure out how to do this while enclosing the svn comment in single quotes which is necessary in order to get the new line in the subversion comment log. I am pulling my hair out trying to figure out how to properly quote and assemble this command. Any help appreciated.

Answer 1

Put your newline in a variable, use the variable wherever you need a newline and change the quotes around your larger string to double quotes. You should also always quote any variable that contains a filename.

nl=$'\n'
svn ci -m"Added new file: ${oFileName}${nl}Orig loc: ${oFileFull}" "${oFileName}"

Answer 2

@Dennis:

Wow, this is hard to wrap my head around, I'd love it if someone could point me towards an online resource that explains this clearly and concisely. The ones I have found have not cleared it up for me.

My latest source of confusion is the following:

#!/bin/bash
nl=$'\n'
msg="Line 1${nl}Line 2"
echo $msg        # ouput = Line 1 Line 2
echo -e $msg     # ouput = Line 1 Line 2
echo "$msg"      # output = Line 1
                 # Line 2

What I'm attempting to illustrate is that having the double quotes around the variable $msg splits the output into two lines, without the double quotes, even with the -e switch, there is no new line.

I don't get why the double quotes are necessary - why isn't the $nl variable expanded when it is assigned as part of the msg variable?

I hope I'm not committing a StackOverflow faux pas by answering my own question. And in fact, I am not really providing an answer, but merely a response to your comment. I cannot format a comment as necessary.

Answer 3

@Scott, I know this has already been answered, but here are some comments on your additional question about the difference between echo $msg and echo "$msg". Your example:

nl=$'\n'
msg="Line 1${nl}Line 2"
echo $msg        # ouput = Line 1 Line 2
echo -e $msg     # ouput = Line 1 Line 2
echo "$msg"      # output = Line 1
                 # Line 2

It really helps to know how many arguments the shell is passing to the echo command in each case. Echo takes any number of arguments. It writes all of them to standard output, in order, with a single space after each one except the last one.

In the first example, echo $msg, the shell replaces $msg with the characters that you've set it to (one of which is a newline). Then -- before invoking echo -- it splits that string into multiple arguments using the Internal Field Separator (IFS), and it passes that resulting list of arguments to echo. By default, the IFS is set to whitespace (spaces, tabs, and newlines), so the shell chops your $msg string into four arguments: "Line", "1", "Line", and "2". Echo never even sees the newline, because the shell considers it a separator, just like a space.

In the second example, echo -e $msg, the echo command receives five arguments: "-e", "Line", "1", "Line", "2". The -e argument tells the shell to expand any backslash characters that it sees in its arguments, but there are none, so echo leaves the arguments unchanged. The result is the same as the first example.

In the last example, echo "$msg", you're telling the shell to treat everything between the double quotation marks as one argument, so echo receives one argument, a string that contains letters, numbers, spaces and newlines. It echoes that string exactly as it receives it, so "Line 1" and "Line 2" appear on separate lines.

Try this: set the IFS to something else and watch how the shell splits your string differently:

nl=$'\n'
msg="Line 1${nl}Line 2"
IFS=i
echo $msg        # ouput = L ne 1
                 # L ne 2

Here, because of the weird IFS, the shell treats the letter i as a separator but not the spaces or newlines. So echo receives three arguments: "L", "ne \nL", and "ne 2". (I'm showing the newline character as \n, but it's really just a single character, like X or p or 5.)

Once you realize that single and double quotation marks on the shell's command line are all about constructing arguments for the commands, the logic begins to make sense.

Answer 4

Bash will concatenate adjacent quoted sections. So for example

$ r=rooty
$ t=tooty
$ echo $r$'\n'$t
rooty
tooty
$ echo "$r $r"$'\n'$t
rooty rooty
tooty

For an online guide, the advanced bash scripting guide may be useful, though it may be no better than just reading bash's long and detailed manual. I recommend using vim's Man command (standard under version 7.2, if not earlier versions) to bring up the bash manual, and then using indent-based folding so that you can browse its heirarchy. Well, assuming that you don't have the time and patience to read all 3300 lines from start to finish.