MapReduce Distributed Cache

Question

Hi,

I am adding a file to distributed cache of Hadoop using

     Configuration cng=new Configuration();
     JobConf conf = new JobConf(cng, Driver.class);
     DistributedCache.addCacheFile(new Path("DCache/Orders.txt").toUri(), cng);

where DCache/Orders.txt is the file in HDFS.

When I try to retrieve this file from the cache in configure method of mapper using:

    Path[] cacheFiles=DistributedCache.getLocalCacheFiles(conf);

I get null pointer. What can be the error?

Thanks

Answer 1

DistributedCache doesn't work in single node mode, it just returns a null pointer. Or at least that was my experience with the current version.

I think the url is supposed to start with the hdfs identifier.

http://hadoop.apache.org/common/docs/current/mapred_tutorial.html#DistributedCache