Classnotfound exception while running hadoop

Question

Hi,

I am new to hadoop.

I have a file Wordcount.java which refers hadoop.jar and stanford-parser.jar

I am running the following commnad

javac -classpath .:hadoop-0.20.1-core.jar:stanford-parser.jar -d ep WordCount.java 

jar cvf ep.jar -C ep .

bin/hadoop jar ep.jar WordCount gutenburg gutenburg1

After executing i am getting the following error:

lang.ClassNotFoundException: edu.stanford.nlp.parser.lexparser.LexicalizedParser

The class is in stanford-parser.jar ...

What can be the possible problem?

Thanks

Answer 1

I think you need to add the standford-parser jar when invoking hadoop also, not just the compiler. (If you look in ep.jar, I imagine it will only have one file in it - WordCount.class)

E.g.

bin/hadoop jar ep.jar WordCount -libjars stanford-parser.jar gutenburg gutenburg1

See Map/Reduce Tutorial

Answer 2

Another option you can try since the -libjars doesn't seem to be working for you is to package everything into a single jar, ie your code + the dependencies into a single jar.

This was how it had to be done prior to ~Hadoop-0.18.0 (somewhere around there they fixed this).

Using ant (i use ant in eclipse) you can set up a build that unpacks the dependencies and adds them to the target build project. You can probably hack this yourself though, by manually unpacking the dependency jar and adding the contents to your jar.

Even though I use 0.20.1 now I still use this method. It makes starting a job form the command-line simpler.

Answer 3

mdma is on the right track, but you'll also need your job driver to implement Tool.