Why my bash can't execute R script ?

Question

My script use an access to mysql to get command arguments to launch Rscript. Its use is as follows : Rscript $RFILE $ARGUMENTS (RFILE corresponding to path to Rscript, and ARGUMENTS corresponding to path file used and agr).

I try, different way, but I still have errors, here a copy of my bash script :

#!/usr/bin/env bash
# Execute R process
# -----------------
### Mysql Setup ###
USER=...
PASS=...
HOST=...
DB=...

# Get Job ID process
# Use to retrieve args in my DB
ID=$1

# Get script name
RFILE=$(mysql -u$USER -p$PASS -e "SELECT script_name FROM JobProcess WHERE script_run_id=$ID;" $DB)
SUBSTRING="script_name"
RFILE="${RFILE//$SUBSTRING}"

# Get script_args
ARGUMENTS=$(mysql -u$USER -p$PASS -e "SELECT script_args FROM JobProcess WHERE script_run_id=$ID;" $DB)
SUBSTRING2="script_args"
ARGUMENTS="${ARGUMENTS//$SUBSTRING2}"

RUN="Rscript $RFILE $ARGUMENTS"

# Try Different execute process
Rscript $RFILE $ARGUMENTS
#eval "$RUN"
#`Rscript $RFILE $ARGUMENTS`
#$RUN

I verified my command line (via echo), and if I made a copy-paste to my shell I can run my R script. But from my bash, I can't execute my script (but command line is good).

By using : Rscript $RFILE $ARGUMENTS, Rscript $RFILE $ARGUMENTS and $RUN, I have this error :

Error in parse(text = args[[i]]) : 
  unexpected end of input in ""path_in='/Users/GR/web-app/Rproject/Inputs/Rscript/Gene-level"
Calls: eval -> parse
Execution halted

By using : eval "$RUN", I have this error :

/Users/GR/web-app/Rproject/Scripts/Rscript.sh: line 38: /Users/GR/web-app/Rproject/Scripts/arg_file_test.R: Permission denied
/Users/GR/web-app/Rproject/Scripts/Rscript.sh: line 44: path_in<-"/Users/GR/web-app/Rproject/Inputs/Rscript/Gene-level Description for Modules.csv": No such file or directory

If I try this in my shell script, all works fine :

SCRIPT="/Users/GR/web-app/Rproject/Scripts/arg_file_test.R"
FILE1="path_in='/Users/GR/web-app/Rproject/Inputs/Rscript/Gene-level Description for Modules.csv'"
FILE2="path_in2='/Users/GR/web-app/Rproject/Inputs/Rscript/Template_Auto.csv'"
FILES="\"$FILE1\" \"$FILE2\""
ARG="l=32 w=33"
RUN="Rscript $SCRIPT $FILES $ARG"

Someone has an idea ?

Thanks

Answer 1

In the sample output from Rscript you quoted:

Error in parse(text = args[[i]]) : 
  unexpected end of input in ""path_in='/Users/GR/web-app/Rproject/Inputs/Rscript/Gene-level"

I notice that there's very odd quoting going on there: two "s at the start, a single ' after the =, and just a single " on the end.

At the very least I'd expect a matching ' at the end of the path, and probably something to balance those "s as well

I'm guessing it's related to this line:

RFILE="${RFILE//$SUBSTRING}"

I'm guessing that you're removing one of the 's in that substitution.

Updated: As others have pointed out, your filenames contain spaces. always put "s around the name of a variable name when it contains filenames, especially when you know those filenames contain spaces. Try:

Rscript "$RFILE" $ARGUMENTS

Answer 2

It appears that you are losing some of the quoting of spaces somewhere. You have this error message:

Error in parse(text = args[[i]]) : 
  unexpected end of input in ""path_in='/Users/GR/web-app/Rproject/Inputs/Rscript/Gene-level"

but from other texts the file name should evidently be

/Users/GR/web-app/Rproject/Inputs/Rscript/Gene-level Description for Modules.csv

This would also make sense as an explanation for your problem, as the variable substitution you are doing could well cause a space-containing argument to lose its protective quoting. Would it be possible for you to rename that file (and any other similar ones) to a name that does not contain spaces, say, by replacing spaces with underscores, and trying again?

Answer 3

Why is this a bash script when you have Rscript? So why don't you rewrite this as an R script executed by Rscript.exe allowing you to test the components

• intialization
• database connection
• core work
• ...

individually?

Edit (in response to your comment): R can call R, either packages via library() or directly via source(). You have a debug problem that is comples and you should try to remove some complextity. Moreover, R scripts can use the getopt or optparse packages to deal with command-line arguments.

Edit 2: Are you aware that R has an RMySQL package that would allow you to call the db from R?

Answer 4

First you may want to try adding -s (updating -e to -se) to your mysql line:

RFILE=$(mysql -u$USER -p$PASS -se "SELECT script_name FROM JobProcess WHERE script_run_id=$ID;" $DB)

You should be able to remove your SUBSTRING replacement. Same goes for your variables ARGUMENTS. Replace -e with -se in your mysql line and remove the SUBSTRING replace.

This probably won't fix your issue but it will remove any questions about your expansion replacement. Although if for some reason there was an \n or something at the end of the first line, where you replace SUBSTRING in $RFILE and $ARGUMENTS...

Answer 5

I put a copy of script fix, but I still have some questions :

#!/usr/bin/env bash
# Execute R process
# -----------------
### Mysql Setup ###
USER=...
PASS=...
HOST=...
DB=...

# Get Job ID process
# Use to retrieve args in my DB
ID=$1

# Get script name
RFILE=$(mysql -u$USER -p$PASS -se "SELECT script_name FROM JobProcess WHERE script_run_id=$ID;" $DB)

# Get script_args
ARGUMENTS=$(mysql -u$USER -p$PASS -se "SELECT script_args FROM JobProcess WHERE script_run_id=$ID;" $DB)

RUN="Rscript $RFILE $ARGUMENTS"
eval "$RUN"

Actually, I can't use (probably due to bad seperated token) : Rscript $RFILE $ARGUMENTS, or $RUN.

The purpose of this script is to run any R script by recovering all the parameters in database. Here a copy of data in my DB :

script_run_id : 161
script_name : /Users/GR/web-app/Rproject/Scripts/arg_file_test.R
script_args : 'path_in<-"/Users/GR/web-app/Rproject/Inputs/Rscript/Gene.csv"' 'path_in2<-"/Users/GR/web-app/Rproject/Inputs/Rscript/Template_Auto.csv"' l=0 w=0 

Part script_name define R script to call, and scripts_args define arguments used with this R script. In this example, my script generate this command line :

Rscript /Users/GR/web-app/Rproject/Scripts/arg_file_test.R 'path_in<-"/Users/GR/web-app/Rproject/Inputs/Rscript/Gene.csv"' 'path_in2<-"/Users/GR/web-app/Rproject/Inputs/Rscript/Template_Auto.csv"' l=0 w=0

Is there a better way to proceed ?
For example, define a new column for files path (ex:script_Files) ?
Separate arguments with comma (ex:l=0, w=3, r=6, etc), and split string into array (an example can help me) ?

Thanks
ps : A good way, is probably to use directly Rscript.

Answer 6

okay, I think you may need to change what you are storing in your database.

your script_args field should probably look like this in your db:

path_in=\"/Users/GR/web-app/Rproject/Inputs/Rscript/Gene.csv\" path_in2=\"/Users/GR/web-app/Rproject/Inputs/Rscript/Template_Auto.csv\" l=0 w=0

without any other punctuation or escapes... When you populate your variable and then expand it during your eval or other method it would look like this after expansion:

Rscript /Users/GR/web-app/Rproject/Scripts/arg_file_test.R path_in="/Users/GR/web-app/Rproject/Inputs/Rscript/Gene.csv" path_in2="/Users/GR/web-app/Rproject/Inputs/Rscript/Template_Auto.csv" l=0 w=0

Also you probably don't need eval in this case and you could run $RUN by itself without eval or even just put

Rscript $RFILE $ARGUMENTS

on a line all by itself.