views:

4750

answers:

36

Shell scripts are often used as glue, for automation and simple one-off tasks. What are some of your favorite "hidden" features of the Bash shell/scripting language?

  • One feature per answer
  • Give an example and short description of the feature, not just a link to documentation
  • Label the feature using bold title as the first line

See also:

+6  A: 

Using Infix Boolean Operators

Consider the simple if:

if [ 2 -lt 3 ]
    then echo "Numbers are still good!"
fi

That -lt looks kinda ugly. Not very modern. If you use double brackets around your boolean expression you can the normal boolean operators!

if [[ 2 < 3 ]]
    then echo "Numbers are still good!"
fi
Patrick
This is not a feature of Bash, but an external program: yes, '[[' is a stand-alone program.
Mathieu Garstecki
madmath: I think you'll find that [ is usually a symlink or hardlink to test, while [[ is a shell built-in. It needs to be parsed by the shell otherwise < looks like input redirection.
Greg Hewgill
No, '[' is a stand alone program. '[[' is not
Vinko Vrsalovic
$ type [[[[ is a shell keyword$ which [[$ # No output
Mark A. Nicolosi
Apparently SO doesn't like newlines in comments, hopefully that isn't too hard to parse. That was on Ubuntu with Bash 3.2.39, BTW.
Mark A. Nicolosi
`[[` is a shell builtin (the `<` wouldn't work with a real command as it would be interpreted as input redirection). `[`/`test` is an external program, but overwritten in bash by it's own builtin.
Marian
+5  A: 

Using arithmetic:

if [[ $((2+1)) = $((1+2)) ]]
    then echo "still ok"
fi
Vinko Vrsalovic
It's amazing how many people don't know this, and use expr in their scripts.
Mark Baker
radoulov
+4  A: 

I recently read Csh Programming Considered Harmful which contained this astounding gem:

Consider the pipeline:

A | B | C

You want to know the status of C, well, that's easy: it's in $?, or $status in csh. But if you want it from A, you're out of luck -- if you're in the csh, that is. In the Bourne shell, you can get it, although doing so is a bit tricky. Here's something I had to do where I ran dd's stderr into a grep -v pipe to get rid of the records in/out noise, but had to return the dd's exit status, not the grep's:

device=/dev/rmt8
dd_noise='^[0-9]+\+[0-9]+ records (in|out)$'
exec 3>&1
status=`((dd if=$device ibs=64k 2>&1 1>&3 3>&- 4>&-; echo $? >&4) |
 egrep -v "$dd_noise" 1>&2 3>&- 4>&-) 4>&1`
exit $status;
Greg Hewgill
What you want is to use the PIPESTATUS variable, which is an array of the exit statuses of each command in the pipe. ${PIPESTATUS[0]} would be what you want here.
Steve Baker
Steve, I never knew that - post it as an answer here! (I'll upvote it if you do :)
Mark Baker
+16  A: 

The special variable random:

if [[ $(($RANDOM % 6)) = 0 ]]
    then echo "BANG"
else
    echo "Try again"
fi
Vinko Vrsalovic
Ah yes, Баш Roulette.
dreamlax
Kevin
+12  A: 

I like the -x feature, allowing to see what's going on in your script.

bash -x script.sh
stephanea
+27  A: 

Almost everything listed under EXPANSION section in the manual

In particular, parameter expansion:

$ I=foobar
$ echo ${I/oo/aa} #replacement
faabar
$ echo ${I:1:2}   #substring
oo
$ echo ${I%bar}   #trailing substitution
foo
$ echo ${I#foo}   #leading substitution
bar
Vinko Vrsalovic
Nice, so thats how I got %Ix=y% in cmd.exe ... :)
majkinetor
+8  A: 

Arrays:

#!/bin/bash

array[0]="a string"
array[1]="a string with spaces and \"quotation\" marks in it"
array[2]="a string with spaces, \"quotation marks\" and (parenthesis) in it"

echo "There are ${#array[*]} elements in the array."
for n in "${array[@]}"; do
    echo "element = >>${n}<<"
done

More details on arrays (and other advanced bash scripting stuff) can be found in the Advanced Bash-Scripting Guide.

JesperE
+8  A: 

Here two of my favorites:

To check the syntax w/o really executing the script use:

bash -n script.sh

Go back to the last directory (yes I know pushd and popd, but this is quicker)

cd -
André
"cd -" has the advantage of working if you forgot to push a directory onto the stack but still want to go back there.
temp2290
+18  A: 

Get back history commands and arguments

It's possible to selectively access previous commands and arguments using the ! operator. It's very useful when you are working with long paths.

You can check your last commands with history.

You can use previous commands with !<n> being n the index of the command in history, negative numbers count backwards from the last command in history.

ls -l foo bar
touch foo bar
!-2

You can use previous arguments with !:<n>, zero is the command, >= 1 are the arguments.

ls -l foo
touch !:2
cp !:1 bar

And you can combine both with !<n>:<m>

ls -l foo bar
touch !:2 !:3
rm !-2:2 !-2:3
!-3

Another ! special modifiers are:

  • * for all the arguments

    ls -l foo bar
    ls !*
    
  • ^ for the first argument (!1 == !^)

  • $ for the last argument

    ls -l foo bar
    cat !$ > /dev/null
    
Jaime Soriano
The ^R keyboard shortcut is really handy too
Mark Baker
I also like alt-^ (alt-shift-6 on US keyboards). It expands history sequences like !:2 so you can see what a command is going to do before you run it.
Doug
+3  A: 

C style numeric expressions:

let x="RANDOM%2**8"
echo -n "$x = 0b"
for ((i=8; i>=0; i--)); do
  let n="2**i"
  if (( (x&n) == n )); then echo -n "1"
  else echo -n "0"
  fi
done
echo ""
Steve Baker
A: 

Not that is is a hidden feature. I think it dosn't exist, but it would be magic to have a special syntax allowing to work on thinks on distinct machine. Something like

cat file1 > machine2:file1

that would copy file1 to you directory on machine2. You can also imagine running programs on distinct machines. It would be something like a hidden ssh connection.

stephanea
I wonder what scp could be for...
Adriano Varoli Piazza
Was the answer by stephanea sarcasm or what?
Vinko Vrsalovic
or just pipe it over ssh
Redbeard 0x0A
ssh machine2 "cat > file1" < file1
niXar
This topic is for existing features, sorry. And scp and ssh are your friends.
HMage
+4  A: 

Truncate content of a file (zeroing file)

> file

Specifically, this is very good for truncating log files, when the file is open by another process, which still may write to the file.

Thevs
+12  A: 

Regular expression handling

Recent bash releases feature regular expression matching, so you can do:

if [[ "mystring" =~ REGEX ]] ; then  
    echo match
fi

where REGEX is a raw regular expression in the format described by man re_format.

Matches from any bracketed parts are stored in the BASH_REMATCH array, starting at element 1 (element 0 is the matched string in its entirety), so you can use this to do regex-powered parsing too.

th_in_gs
It feels kinda weird not having to enclose the regex in quotes . . .
dreamlax
A: 

Embedded Command substitution:

hostname && dig +short $(hostname) && dig +short -x $(dig +short $(hostname))

This command is good for checking RDNS on your mail server. :P

Nathacof
Signatures are frowned upon.
Dennis Williamson
+38  A: 

insert preceding line's final parameter

alt-. the most useful key combination ever, try it and see, for some reason no one knows about this one.

press it again and again to select older last parameters.

great when you want to do something else to something you used just a moment ago.

chillitom
Definetly +1. Thanks for this one, so useful yet so hidden.
Alberto Zaccagni
Can I use Alt+. to give you +2 ?
Adam Liss
I'm a frequent user of !$, but this is far more immediate and useful.
jmanning2k
I find `!$` too hard to type quickly. I always have to slow down and think about putting the dollar sign second. `Alt`+`.` is faster and easier. Not to mention, you actually get to see the text before you execute it.
dreamlax
+23  A: 

If you want to keep a process running after you log out:

disown -h <pid>

is a useful bash built-in. Unlike nohup, you can run disown on an already-running process.

First, stop your job with control-Z, get the pid from ps (or use echo $!), use bg to send it to the background, then use disown with the -h flag.

Don't forget to background your job or it will be killed when you logout.

Alex Reynolds
That is sweet! So many times I've wanted to do that. Can you also redirect outputs afterwards?
razzed
Better get the PID from `jobs -l` (or `-p`)
Marian
+9  A: 

Quick & Dirty correction of typos (especially useful for long commands over slow connections where using the command history and scrolling through it would be horrible):

$ cat /proc/cupinfo
cat: /proc/cupinfo: No such file or directory
$ ^cup^cpu

Also try !:s/old/new which substitutes old with new in the previous command once.

If you want to substitute many occurrences you can do a global substitution with !:gs/old/new.

You can use the gs and s commands with any history event, e.g.

!-2:s/old/new

To substitute old with new (once) in the second to last command.

mihi
is there any way to find more about this or similar features? googling for ^foo^bar is not that satisfying :)
Tetha
man bash? :) (or info bash)
mihi
Event Designators and Modifiers in `man bash`. Try `!:s/old/new` which substitutes old with new in the previous command once. If you want to substitute many occurrences you can do a global substitution with `!:gs/old/new`.This may be combined with James' post (http://stackoverflow.com/questions/211378/hidden-features-of-bash/211913#211913), e.g.:`!$:s/old/new` (substitutes old with new in the last argument of the previous command), `!-2:0:gs/a/s !-2*` (substitutes every occurrence of a with s in the penultimate command name and adds all the arguments of the penultimate command). Good luck!
Baldur
+11  A: 

Here is one of my favorites. This sets tab completion to not be case sensitive. It's really great for quickly typing directory paths, especially on a mac where the file system is not case sensitive by default. I put this in .inputrc in my home folder.

set completion-ignore-case on
qbn
+6  A: 

Running a command before displaying the bash prompt

Set a command in the "PROMPT_COMMAND" env variable and it will be run automatically before each prompt. Example:

[[email protected]]$ export PROMPT_COMMAND="date"
Fri Jun  5 15:19:18 BST 2009
[[email protected]]$ ls
file_a  file_b  file_c
Fri Jun  5 15:19:19 BST 2009
[[email protected]]$ ls

For the next april fools, add "export PROMPT_COMMAND=cd" to someone's .bashrc then sit back and watch the confusion unfold.

lsc
+12  A: 

SECONDS=0; sleep 5 ; echo "that took approximately $SECONDS seconds"

SECONDS

Each time this parameter is referenced, the number of seconds since shell invocation is returned. If a value is assigned to SECONDS, the value returned upon subsequent references is the number of seconds since the assignment plus the value assigned. If SECONDS is unset, it loses its special properties, even if it is subsequently reset.

Alberto Zaccagni
+1  A: 

Using 'let' built-in bash command for basic arithmetic

A=10
let B="A * 10 + 1" # B=101
let B="B / 8"      # B=12, let does not do floating point
let B="(RANDOM % 6) + 1" # B is now a random number between 1 and 6

To do floating point evaluations, you can use the "bc" command (no part of bash).

FP=`echo "scale=4; 10 / 3" | bc` # FP="3.3333"
lsc
There's also `$(( ))` for arithmetic expansion and `(( ))` for tests
Daenyth
+2  A: 

These properties are another one of my favorites.

export HISTCONTROL=erasedups
export HISTSIZE=1000

The first one makes sure bash doesn't log commands more than once, will really improves history's usefulness. The other expands the history size to 1000 from the default of 100. I actually set this to 10000 on my machines.

qbn
+6  A: 

Ctrlx Ctrle

This will load the current command into the editor defined in the variable VISUAL. This is really useful for long commands like some of those listed here.

To use vi as your editor:

export VISUAL=vi
Robin
Wow. Very useful!
razzed
`set -o vi` then Esc on a command goes to inline editing, a plain 'v' pulls the command into a full vi editor.
Stephen P
+1  A: 

Easily move around between multiple directories

Not a hidden feature, but much more flexible than pushd which requires stack-like navigation.

a() { alias $1=cd\ $PWD; }

cd somewhere and type "a 1". Later on just typing "1" will return to that directory.

David Plumpton
+15  A: 

My favorite:

sudo !!

Rerun the previous command with sudo.

GloryFish
This is a special case of http://stackoverflow.com/questions/211378/hidden-features-of-bash/211913#211913
Vinko Vrsalovic
It's almost like saying **DO IT!!**
dreamlax
+1  A: 

One I use a lot is !$ to refer to the last word of the last command:

$ less foobar.txt
...
# I dont want that file any more
$ rm !$
camh
You can also do `Alt + .` for the same
Daenyth
That depends on what line edit mode you have. In vi-mode (set -o vi) `Alt + .` does not seem to work.
camh
+4  A: 
Adam Liss
+6  A: 
Adam Liss
+2  A: 

As others have mentioned, Ctrl-r is great for stepping back through your command history. But what if you want to go forward after you've taken one or a few steps too many? That's where Ctrl-s comes in handy. However, it's normally mapped to XOFF (interrupt data flow). Since that's not too useful any more because we're not using slow serial terminals, you can turn off that mapping with:

stty -ixon

in your ~/.bashrc file.

This also makes Ctrl-q available which is normally a duplicate of Ctrl-v (quoted-insert which allows you to insert a literal control character). I have Ctrl-q mapped to menu-complete which steps through completions when pressed repeatedly. I like to leave Tab set to regular complete.

You can set Ctrl-q to menu-complete by adding this line to your ~/.inputrc file:

"\C-q": menu-complete
Dennis Williamson
A: 

Bash has variable indirection:

$ foo=bar
$ baz=foo
$ echo ${!baz}
bar
Dennis Williamson
+1  A: 

I have an alias r='fc-s', and I find it very useful in some limited cases. To run the last command, just type r and hit enter, and that's it. Of course, that itself is not very useful because up arrow does the same thing. But you can use r to run the previous command with substitutions. Let's say your last command was a long command compiling some file:

$ gcc -c <file_name>.c <lots of options> -o <file_name>.o

Now you want to compile another file with the same options and have a corresponding .o file:

$ r <file_name>=<new_file>

will do it. You don't have to use up arrow, navigate to the right places and then replace them each manually. This can be repeated multiple times, so you can do this next:

$ r <new_file>=<other_file>

Of course, for such a thing you have makefiles, but I hope I have shown that the alias is useful.

I haven't needed the use of this alias a lot, but there have been times that I have been glad that I have this alias!

Alok
+4  A: 

export TMOUT=$((15*60))

Terminate bash after 15 minutes of idle time, set to 0 to disable. I usually put this to ~/.bashrc on my root accounts. It's handy when administrating your boxes and you may forget to logout before walking away from the terminal.

Mike Nelson
+3  A: 

Not really a feature but rather a direction: I found many "hidden features", secrets and various bash usefulness at commandlinefu.com. Many of the highest rated answers to this answers, I learned them on that site :)

Agos
+1  A: 

set -o vi in order to have vi-like editing of the command history as well as of the currently typed command.

René Nyffenegger
+2  A: 

Undo

C-S-- Control Shift Minus Undo-es typing actions.

Kill / Yank

Any delete operation C-w (delete previous word), C-k (delete to end of line), C-u (delete to start of line) etc... copies it's deleted text to the kill ring, you can paste the last kill with: C-y and cycle through (and paste from) the ring of deleted items with Alt-y

slomojo
+3  A: 

You can ignore certain files while tab completing by setting th FIGNORE variable.

For example, if you have a subverion repo and you want to navigate more easily do

export FIGNORE=".svn"

now you can cd without being blocked by .svn directories.

Sionide21