URLEncode from a bash script

Question

I am trying to write a bash script for testing that takes a parameter and sends it through curl to web site. I need to url encode the value to make sure that special characters are processed properly. What is the best way to do this?

Here is my basic script so far:

#!/bin/bash
host=${1:?'bad host'}
value=$2
shift
shift
curl -v -d "param=${value}" http://${host}/somepath $@

Answer 1

I knew I'd seen how to do it: http://andy.wordpress.com/2008/09/17/urlencode-in-bash-with-perl/

Answer 2

If you don't want to depend on Perl you can also use sed:

http://www.unix.com/shell-programming-scripting/59936-url-encoding.html

Answer 3

Use Perl's URI::Escape module and uri_escape function in the second line of your bash script:

...

value="$(perl -MURI::Escape -e 'print uri_escape($ARGV[0]);' "$2")"
...

Edit: Fix quoting problems, as suggested by Chris Johnsen in the comments. Thanks!

Answer 4

Direct link to awk version : http://www.shelldorado.com/scripts/cmds/urlencode
I used it for years and it works like a charm

Answer 5

I've found the following snippet useful to stick it into a chain of program calls, where URI::Escape might not be installed:

perl -p -e 's/([^A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg'

(via)

Answer 6

Or just use curl --data-urlencode

Answer 7

I find it more readable in python:

encoded_value=$(python -c "import urllib; print urllib.quote('''$value''')")

the triple ' ensures that single quotes in value won't hurt. urllib is in the standard library. It work for exampple for this crazy (real world) url:

"http://www.rai.it/dl/audio/" "1264165523944Ho servito il re d'Inghilterra - Puntata 7

Answer 8

@sandro thkz! I need that

Answer 9

For those of you looking for a solution that doesn't need perl, here is one that only needs hexdump and awk:

url_encode() {
 [ $# -lt 1 ] && { return; }

 encodedurl="$1";

 # make sure hexdump exists, if not, just give back the url
 [ ! -x "/usr/bin/hexdump" ] && { return; }

 encodedurl=`
   echo $encodedurl | hexdump -v -e '1/1 "%02x\t"' -e '1/1 "%_c\n"' |
   LANG=C awk '
     $1 == "20"                    { printf("%s",   "+"); next } # space becomes plus
     $1 ~  /0[adAD]/               {                      next } # strip newlines
     $2 ~  /^[a-zA-Z0-9.*()\/-]$/  { printf("%s",   $2);  next } # pass through what we can
                                   { printf("%%%s", $1)        } # take hex value of everything else
   '`

}

Stitched together from a couple of places across the net and some local trial and error. It works great!