error in reading R file while submitting a r job to condor

Question

Hi Everyone,

I have a R Job that is submitted to the condor, The R file(one.R) which is submitted to the condor is reading another R file(two.R), when I submit the job to the condor its is failed and the reason for that is the submitted R(one.R) file is not reading the called R file(two.R) Error in text file is :

Error in file(file, "rt") : cannot open the connection
Calls: read.table -> file
In addition: Warning message:
In file(file, "rt") :
  cannot open file 'C:/Users/pcname/Desktop/test_case/two.R': Permission denied
Execution halted 

and my submit file is

#test_input.condor
#

executable = C:\R\R-2.10.1\bin\Rscript.exe
arguments = one.R
universe = vanilla
getenv = true
#requirements = ARCH == "INTEL" && OPSYS == "WINNT60" 
input = one.R

should_transfer_files = yes
transfer_executable = false
when_to_transfer_output = ON_EXIT
transfer_input_files = C:/Users/OmegaAdmin/Desktop/test_case/two.R

log = test_input.log
output = test_input.out
error = test_input.err

queue 

Appreciate any ideas on this.

Thanks,

Answer 1

This is not an R-related problem, but a problem of accessibility. The error message seems rather clear to me: the server has no reading rights for that file. Make sure you share the file or folder you want to read in. I don't know what the setup is of the network and clusters wherever you're located, but you better contact the admins to ask how you get your files to the right places.

Also make sure that if you transfer files to the servers/cluster, you adapt your R script so it points to the right directories. Which is probably not your own harddrive...

Answer 2

When you say

transfer_input_files = C:/Users/OmegaAdmin/Desktop/test_case/two.R

That tells Condor to copy two.R into the current working directory when the job starts. The current working directory is a specially created workspace, not (usually) a home directory. Thus I would expect the full path to look something like

C:/condor/execute/dir_28412/two.R

However, R is actually looking in

C:/Users/pcname/Desktop/test_case/two.R

Why is R looking there? Does one.R potentially say "Find two.R in $HOME/Desktop/test_case"? Does it perhaps say, "Look in Desktop/testcase/two.R" and R has configuration that wants to look relative to the user's home directory?

The solution is almost certainly to modify one.R or your R configuration to look for two.R in the current working directory. If for some reason R changes its current working directory, the environment variable _CONDOR_SCRATCH_DIR should contain it.

---

On a related note, you said:

arguments = one.R
input = one.R

The first is an argument passed to Rscript.exe, which I'm guessing tells R to load and run a file called one.R. Except that script isn't present! If you want that to work, you'll need to add it to transfer_input_files. But it obviously appears to work; why? Because "input=one.R" means "take the contents of one.R and pipe it in as standard input to Rscript.exe; the same as if you'd typed those contents in. I'm guessing you can remove the arguments, or remove the input and add one.R to your transfer_input_files, removing the ambiguity.