Bash, curl, eval and whitespace

Question

I'm writing a script to execute CURL commands for a given user input. The script has multiple helper function to create the list of parameters (arguments) that will eventually be passed to CURL.

A stripped out example, is as follows :

#!/bin/bash
function create_arg_list
{
    # THIS HTTP HEADER VALUE COMES FROM THE USER and MAY CONTAIN SPACES
 local __hdr="x-madhurt-test:madh urt"
 local __params="http://google.co.in -X GET -H '$__hdr'"

 local __resultvar="$1"
 eval $__resultvar="\"$__params\""
 echo "params after eval : $__resultvar"
}


create_arg_list arg_list
echo "params after eval in main code : $arg_list"

echo "Running command : curl -v $arg_list"
curl -v $arg_list

The script works great when the input parameters (file path, url etc..) have (quoted) white space in them. However, when the arguments that are supposed to be passed as HTTP Headers to CURL contain spaces, the script fails miserably.

Here is what I've tried :

1. Use single quotes around the header value (e.g. '$__hdr'). However, with this the value that is passed to CURL is :

   curl -v http://google.co.in -X GET -H 'x-madhurt-test:madh urt'

   , which is treated as-as by CURL and the actual header that is sent is like this :

   'x-madhurt-test:madh

2. Double escape the header value (e.g. \\"$__hdr\\"), but this does seem to work as well. In this case CURL gets "urt" as a separate parameter and treats it as a URL

   curl: (6) Couldn't resolve host 'urt"'

3. Escape the space in the header value (i.e. use "madh\ urt" instead of "madh urt"), but this turns out to be the same as option 2.

Does someone have insights as to what is happening wrong here?

Answer 1

This code works but it's not intended to use as-is. I'm posting it to give you some ideas for how you might proceed. They key to making what you want to do work is to use an array. Unfortunately, Bash can't return arrays from functions. What you probably ought to do is use a global array. The code below, however, creates a string out of a declare statement and passes that through your indirect variable. It's a seriously bad kludge.

#!/bin/bash
function create_arg_list
{
    # THIS HTTP HEADER VALUE COMES FROM THE USER and MAY CONTAIN SPACES
 local __hdr="x-madhurt-test:madh urt"
 local __params="http://google.co.in -X GET -H"
 __params=($__params "$__hdr")

 local __resultvar="$1"
 eval $__resultvar=$(printf "%q" "$(declare -p __params)")
 echo "params after eval : $__resultvar"
}

create_arg_list arg_list
echo "params after eval in main code : $arg_list"

echo "Running command : curl -v $arg_list"

eval $arg_list

curl -v "${__params[@]}"

Answer 2

Dennis's answer is good, so I'll focus on why your code does not work. Let's use an auxiliar function to show the arguments received by a function:

$ args() { for x in "$@"; do echo $x; done }
$ args 1 '2 b' 3
1
2 b
3

Ok, as expected the quotes are solely used to delimit arguments. But now let's use a variable as you did:

$ var="1 '2 b' 3"
$ args $var
1
'2
b'
3

Bash expands the variable and the function (or command) gets the quotes. That's not what you wanted, of course.

Solution1: Use eval to re-interpret quotes.

$ eval args $var
1
2 b
3

Solution2: Use an array and expand it with "${MYARRAY[@]}" as Dennis showed.

More ideas: a technique I've sometimes used is doing the eval outside:

$ eval "$(create_args_list VARNAME)"

In this case otherfun would return a string that, when evaled, would create a variable with name VARNAME (that can be even local). This variable (or variables, if needed) can be string or arrays. Again, I'd use an array so it can be easily used afterwards:

$ curl "${VARNAME[@]}"