Hi, I am using hadoop, and working on my map and reduce tasks, where I need to be able to access configuration files that are stored in a named folder. I also have a folder with jars that map must call using process builder, as these jars don't belong to me.
At this point I am storing the files in nfs, where all the hadoop nodes can access them using the same path.
CmdFolder="/mnt/data/fwang/akintayo/jars/";
CmdSettingsFolder="/mnt/data/fwang/akintayo/settings/";
String CmdJar = "extract.jar";
String CmdArray[] = new String[4];
CmdArray[0]="java";
CmdArray[1]="-jar";
CmdArray[2]=CmdFolder+CmdJar;
CmdArray[3]=signalName;
ProcessBuilder procBuilder = new ProcessBuilder(CmdArray);
I take a similar approach with the setting, except that the I merely pass the folder and the jars find their specific configuration file.
This works, but it seems like a very inefficient approach. Is there a way for me to efficiently access the contents of a folder?
Thanks