views:

2008

answers:

16

Or more generally, how do I remove an item from a colon-separated list in a Bash environment variable?

I thought I had seen a simple way to do this years ago, using the more advanced forms of Bash variable expansion, but if so I've lost track of it. A quick search of Google turned up surprisingly few relevant results and none that I would call "simple" or "elegant". For example, two methods using sed and awk, respectively:

PATH=$(echo $PATH | sed -e 's;:\?/home/user/bin;;' -e 's;/home/user/bin:\?;;')
PATH=!(awk -F: '{for(i=1;i<=NF;i++){if(!($i in a)){a[$i];printf s$i;s=":"}}}'<<<$PATH)

Does nothing straightforward exist? Is there anything analogous to a split() function in Bash?

Update:
It looks like I need to apologize for my intentionally-vague question; I was less interested in solving a specific use-case than in provoking good discussion. Fortunately, I got it!

There are some very clever techniques here, but it looks like there's no way to accomplish this using pure Bash script (though nicerobot's IFS/array technique comes pleasantly close). In the end, I've added the following three functions to my toolbox. The magic happens in path_remove, which is based largely on Martin York's clever use of awk's RS variable.

path_append ()  { path_remove $1; export PATH="$PATH:$1"; }
path_prepend () { path_remove $1; export PATH="$1:$PATH"; }
path_remove ()  { export PATH=`echo -n $PATH | awk -v RS=: -v ORS=: '$0 != "'$1'"' | sed 's/:$//'`; }

The only real cruft in there is the use of sed to remove the trailing colon. Considering how straightforward the rest of Martin's solution is, though, I'm quite willing to live with it!


Related question: http://stackoverflow.com/questions/273909/how-do-i-manipulate-path-elements-in-shell-scripts

+2  A: 

Well, in bash, as it supports regular expression, I would simply do :

PATH=${PATH/:\/home\/user\/bin/}
mat
Isn't it only pathname expansion, not regular expressions?
dreamlax
While bash does support regular expressions (as of bash 3), this is not an example of it, this is a variable substitution.
Robert Gamble
This is pattern variable expantion and the solution has several problems. 1) it will not match the first element. 2) it will match anything that starts with "/home/user/bin", not just "/home/user/bin". 3) it requires escaping special characters. At best, i'd say this is an incomplete example.
nicerobot
A: 

My dirty hack:

echo ${PATH} > t1
vi t1
export PATH=`cat t1`
Martin York
haha, fine humor :)
Johannes Schaub - litb
It's never a good sign when the most obvious solution is to *de*-automate the process. :-D
Ben Blank
+2  A: 

I did write an answer to this here (using awk too). But i'm not sure that's what you are looking for? It at least looks clear to me what it does, instead of trying to fit into one line. For a simple one liner, though, that only removes stuff, i recommend

echo $PATH | tr ':' '\n' | awk '$0 != "/bin"' | paste -sd:

Replacing is

echo $PATH | tr ':' '\n' | 
    awk '$0 != "/bin"; $0 == "/bin" { print "/bar" }' | paste -sd:

or (shorter but less readable)

echo $PATH | tr ':' '\n' | awk '$0 == "/bin" { print "/bar"; next } 1' | paste -sd:

Anyway, for the same question, and a whole lot of useful answers, see here.

Johannes Schaub - litb
+2  A: 

Since this tends to be quite problematic, as in there IS NO elegant way, I recommend avoiding the problem by rearranging the solution: build your PATH up rather than attempt to tear it down.

I could be more specific if I knew your real problem context. In the interim, I will use a software build as the context.

A common problem with software builds is that it breaks on some machines, ultimately due to how someone has configured their default shell (PATH and other environment variables). The elegant solution is to make your build scripts immune by fully specifying the shell environment. Code your build scripts to set the PATH and other environment variables based on assembling pieces that you control, such as the location of the compiler, libraries, tools, components, etc. Make each configurable item something that you can individually set, verify, and then use appropriately in your script.

For example, I have a Maven-based WebLogic-targeted Java build that I inherited at my new employer. The build script is notorious for being fragile, and another new employee and I spent three weeks (not full time, just here and there, but still many hours) getting it to work on our machines. An essential step was that I took control of the PATH so that I knew exactly which Java, which Maven, and which WebLogic was being invoked. I created environment variables to point to each of those tools, then I calculated the PATH based on those plus a few others. Similar techniques tamed the other configurable settings, until we finally created a reproducible build.

By the way, don't use Maven, Java is okay, and only buy WebLogic if you absolutely need its clustering (but otherwise no, and especially not its proprietary features).

Best wishes.

Rob Williams
+5  A: 

A minute with awk:

# Strip all paths with SDE in them.
#
export PATH=`echo ${PATH} | awk -v RS=: -v ORS=: '/SDE/ {next} {print}'`
Martin York
Or shorter, `PATH$(awk -vRS=: -vORS=: '!/SDE/' <<<$PATH)`.
ephemient
+2  A: 

As with @litb, I contributed an answer to the question "How do I manipulate $PATH elements in shell scripts", so my main answer is there.

The 'split' functionality in bash and other Bourne shell derivatives is most neatly achieved with $IFS, the inter-field separator. For example, to set the positional arguments ($1, $2, ...) to the elements of PATH, use:

set -- $(IFS=":"; echo "$PATH")

It will work OK as long as there are no spaces in $PATH. Making it work for path elements containing spaces is a non-trivial exercise - left for the interested reader. It is probably simpler to deal with it using a scripting language such as Perl.

I also have a script, clnpath, which I use extensively for setting my PATH. I documented it in the answer to "How to keep from duplicating PATH variable in csh".

Jonathan Leffler
IFS=: a=( $PATH ); IFS= splitting is also nice. works if they contain spaces too. but then you got an array, and have to fiddle with for loops and such to remove the names.
Johannes Schaub - litb
Yes; it gets fiddly - as with my updated comment, it is probably simpler to use a scripting language at this point.
Jonathan Leffler
+2  A: 

Here's the simplest solution i can devise:

#!/bin/bash
IFS=:
# convert it to an array
t=($PATH)
unset IFS
# perform any array operations to remove elements from the array
t=(${t[@]%%*usr*})
IFS=:
# output the new array
echo "${t[*]}"

The above example will remove any element in $PATH that contains "usr". You can replace "*usr*" with "/home/user/bin" to remove just that element.

nicerobot
As one liner:PATH=$(IFS=':';t=($PATH);unset IFS;t=(${t[@]%%*usr*});IFS=':';echo "${t[*]}");
nicerobot
A: 

What makes this problem annoying are the fencepost cases among first and last elements. The problem can be elegantly solved by changing IFS and using an array, but I don't know how to re-introduce the colon once the path is converted to array form.

Here is a slightly less elegant version that removes one directory from $PATH using string manipulation only. I have tested it.

#!/bin/bash
#
#   remove_from_path dirname
#
#   removes $1 from user's $PATH

if [ $# -ne 1 ]; then
  echo "Usage: $0 pathname" 1>&2; exit 1;
fi

delendum="$1"
NEWPATH=
xxx="$IFS"
IFS=":"
for i in $PATH ; do
  IFS="$xxx"
  case "$i" in
    "$delendum") ;; # do nothing
    *) [ -z "$NEWPATH" ] && NEWPATH="$i" || NEWPATH="$NEWPATH:$i" ;;
  esac
done

PATH="$NEWPATH"
echo "$PATH"
Norman Ramsey
A: 

Here's a Perl one-liner:

PATH=`perl -e '$a=shift;$_=$ENV{PATH};s#:$a(:)|^$a:|:$a$#$1#;print' /home/usr/bin`

The $a variable gets the path to be removed. The s (substitute) and print commands implicitly operate on the $_ variable.

J. A. Faucett
+1  A: 

Good stuff here. I use this one to keep from adding dupes in the first place.

#!/bin/bash
#
######################################################################################
#
# Allows a list of additions to PATH with no dupes
# 
# Patch code below into your $HOME/.bashrc file or where it
# will be seen at login.
#
# Can also be made executable and run as-is.
#
######################################################################################

# add2path=($HOME/bin .)                  ## uncomment space separated list 
if [ $add2path ]; then                    ## skip if list empty or commented out
for nodup in ${add2path[*]}
do
    case $PATH in                 ## case block thanks to MIKE511
    $nodup:* | *:$nodup:* | *:$nodup ) ;;    ## if found, do nothing
    *) PATH=$PATH:$nodup          ## else, add it to end of PATH or
    esac                          ## *) PATH=$nodup:$PATH   prepend to front
done
export PATH
fi
## debug add2path
echo
echo " PATH == $PATH"
echo
ongoto
A: 

With extended globbing enabled it's possible to do the following:

# delete all /opt/local paths in PATH
shopt -s extglob 
printf "%s\n" "${PATH}" | tr ':' '\n' | nl
printf "%s\n" "${PATH//+(\/opt\/local\/)+([^:])?(:)/}" | tr ':' '\n' | nl 

man bash | less -p extglob
carlo
A: 

Extended globbing one-liner (well, sort of):

path_remove ()  { shopt -s extglob; PATH="${PATH//+(${1})+([^:])?(:)/}"; export PATH="${PATH%:}"; shopt -u extglob; return 0; } 

There seems no need to escape slashes in $1.

path_remove ()  { shopt -s extglob; declare escArg="${1//\//\\/}"; PATH="${PATH//+(${escArg})+([^:])?(:)/}"; export PATH="${PATH%:}"; shopt -u extglob; return 0; } 
carlo
+1  A: 

Since the big issue with substitution is the end cases, how about this:

WORK=:$PATH:
REMOVE=/usr/bin
WORK=${WORK/:$REMOVE:/:}
WORK=${WORK%:}
WORK=${WORK#:}
PATH=$WORK

Pure bash :).

Andrew Aylett
A: 

Yes, putting a colon at the end of PATH, for example, makes removing a path a bit less clumsy & error-prone.

path_remove ()  { 
   declare i newPATH
   newPATH="${PATH}:"
   for ((i=1; i<=${#@}; i++ )); do
      #echo ${@:${i}:1}
      newPATH="${newPATH//${@:${i}:1}:/}" 
   done
   export PATH="${newPATH%:}" 
   return 0; 
} 

path_remove_all ()  {
   declare i newPATH
   shopt -s extglob
   newPATH="${PATH}:"
   for ((i=1; i<=${#@}; i++ )); do
      newPATH="${newPATH//+(${@:${i}:1})*([^:]):/}" 
      #newPATH="${newPATH//+(${@:${i}:1})*([^:])+(:)/}" 
   done
   shopt -u extglob 
   export PATH="${newPATH%:}" 
   return 0 
} 

path_remove /opt/local/bin /usr/local/bin

path_remove_all /opt/local /usr/local 
cyrill
A: 

Adding colons to PATH we could also do something like:

path_remove ()  { 
   declare i newPATH
   # put a colon at the beginning & end AND double each colon in-between
   newPATH=":${PATH//:/::}:"   
   for ((i=1; i<=${#@}; i++)); do
       #echo ${@:${i}:1}
       newPATH="${newPATH//:${@:${i}:1}:/}"   # s/:\/fullpath://g
   done
   newPATH="${newPATH//::/:}"
   newPATH="${newPATH#:}"      # remove leading colon
   newPATH="${newPATH%:}"      # remove trailing colon
   unset PATH 
   PATH="${newPATH}" 
   export PATH
   return 0 
} 


path_remove_all ()  {
   declare i newPATH extglobVar
   extglobVar=0
   # enable extended globbing if necessary
   [[ ! $(shopt -q extglob) ]]  && { shopt -s extglob; extglobVar=1; }
   newPATH=":${PATH}:"
   for ((i=1; i<=${#@}; i++ )); do
      newPATH="${newPATH//:+(${@:${i}:1})*([^:])/}"     # s/:\/path[^:]*//g
   done
   newPATH="${newPATH#:}"      # remove leading colon
   newPATH="${newPATH%:}"      # remove trailing colon
   # disable extended globbing if it was enabled in this function
   [[ $extglobVar -eq 1 ]] && shopt -u extglob
   unset PATH 
   PATH="${newPATH}" 
   export PATH
   return 0 
} 

path_remove /opt/local/bin /usr/local/bin

path_remove_all /opt/local /usr/local 
proxxy
A: 

In path_remove_all (by proxxy):

-newPATH="${newPATH//:+(${@:${i}:1})*([^:])/}" 
+newPATH="${newPATH//:${@:${i}:1}*([^:])/}"        # s/:\/path[^:]*//g 
marius