Java text classification problem

Question

Hello,

I have a set of Books objects, classs Book is defined as following :

Class Book{

String title;
ArrayList<tags> taglist;

}

Where title is the title of the book, example : Javascript for dummies.

and taglist is a list of tags for our example : Javascript, jquery, "web dev", ..

As I said a have a set of books talking about different things : IT, BIOLOGY, HISTORY, ... Each book has a title and a set of tags describing it..

I have to classify automaticaly those books into separated sets by topic, example :

IT BOOKS :

• Java for dummies
• Javascript for dummies
• Learn flash in 30 days
• C++ programming

HISTORY BOOKS :

• World wars
• America in 1960
• Martin luther king's life

BIOLOGY BOOKS :

• ....

Do you guys know a classification algorithm/method to apply for that kind of problems ?

A solution is to use an external API to define the category of the text, but the problem here is that books are in different languages : french, spanish, english ..

Answer 1

You might want to look up fuzzy matching algorithms such as Soundex and Levenshtein.

Answer 2

So you are looking to make a Map of Tags that holds a Collection of Books?

EDIT:

Sounds like you might want to take a look at a Vector Space Model to apply classification of categories.

Either Lucene or Classifier4j offer a framework for this.

Answer 3

This looks like a reasonably straightforward keyword-based classification task. Since you're using Java, good packages to consider for this would be Classifier4J, Weka, or Lucene Mahout.

Classifier4J

Classifier4J supports classification using naive Bayes and a vector space model.

As seen in this source code snippet on training and scoring using its naive Bayes classifier, the package is reasonably easy to use. It's also distributed under the liberal Apache Software License.

Weka

Weka is a very popular tool for data mining. An advantage of using it is that you'd be able to readily experiment with using numerous different machine learning models to categorize the books into topics including naive Bayes, decision trees, support vector machines, k-nearest neighbor, logistic regression, and even a rule set based learner.

You'll find a tutorial on using Weka for text categorization here.

Weka is, however, distributed under the GPL. You won't be able to use it for closed source software that you want to distribute. But, you could still use it to back a web service.

Lucene Mahout

Mahout is designed for doing machine learning on very large datasets. It's built on top of Apache Hadoop and supports supervised classification using naive Bayes.

You'll find a tutorial covering how to use Mahout for text classification here.

Like Classifier4J, Mahout is distributed under the liberal Apache Software License.

Answer 4

Do you not want something as simple as this?

Map<Tag, ArrayList<Book>> m = {};
for (Book b : books) {
    for (tag t : b.taglist) {
        m.get(t).add(b);
    }
}

Now m.get("IT") will return all IT books, etc...

Sure some books will appear in multiple categories, but that happens in real life, too...