Implementing Naïve Bayes algorithm in Java - Need some guidance

Question

hello stackflow people

As a School assignment i'm required to implement Naïve Bayes algorithm which i am intending to do in Java.

In trying to understand how its done, i've read the book "Data Mining - Practical Machine Learning Tools and Techniques" which has a section on this topic but am still unsure on some primary points that are blocking my progress.

Since i'm seeking guidance not solution in here, i'll tell you guys what i thinking in my head, what i think is the correct approach and in return ask for correction/guidance which will very much be appreciated. please note that i am an absolute beginner on Naïve Bayes algorithm, Data mining and in general programming so you might see stupid comments/calculations below:

The training data set i'm given has 4 attributes/features that are numeric and normalized(in range[0 1]) using Weka (no missing values)and one nominal class(yes/no)

1) The data coming from a csv file is numeric HENCE

* Given the attributes are numeric i use PDF (probability density function) formula.
+ To calculate the PDF in java i first separate the attributes based on whether they're in class yes or class no and hold them into different array (array class yes and array class no)
+ Then calculate the mean(sum of the values in row / number of values in that row) and standard divination for each of the 4 attributes (columns) of each class
+ Now to find PDF of a given value(n) i do (n-mean)^2/(2*SD^2),
+ Then to find P( yes | E) and P( no | E) i multiply the PDF value of all 4 given attributes and compare which is larger, which indicates the class it belongs to

In temrs of Java, i'm using ArrayList of ArrayList and Double to store the attribute values.

lastly i'm unsure how to to get new data? Should i ask for input file (like csv) or command prompt and ask for 4 values?

I'll stop here for now (do have more questions) but I'm worried this won't get any responses given how long its got. I will really appreciate for those that give their time reading my problems and comment.

Answer 1

What you are doing is almost correct.

         + Then to find P( yes | E) and P( no | E) i multiply the PDF value of all 4 given attributes and compare which is larger, which indicates the class it belongs to 

Here, you forgot to multiply the prior P(yes) or P(no). Remember the decision formulae:

P(Yes | E) ~= P(Attr_1 | Yes) * P(Attr_2 | Yes) * P(Attr_3 | Yes) * P(Attr_4 | Yes) * P(Yes)

For Naive Bayes (and any other supervised learning/classification algorithms), you need to have training data and testing data. You use training data to train the model and do prediction on the testing data. You could simply use training data as testing data. Or you can split the csv file into two pieces, one for training and one for testing. You could also do cross validation on the csv file.

Answer 2

Thank you for your answer Yin Zhu.

But if you check on my first post all my data is numeric, multiplying by Zero will gives inaccurate result that's why i didn't want to multiply by prior P(Yes) or P(No) cause the class is numeric 0 or 1.(Are you sure i should?)

The Zero attributes are also hammering my progress. For example i know Laplace Correction adds 1 to denominator and 3 to numerator BUT given i'm using PDF, do you still apply Laplace?

I have the mean and Standard deviation of Class P(Yes) and P(No) right, so i assumed using the PDF formula to plugin the value of each attribute would give the wanted value but now not sure and I'm really confused about this because what if the value is 0.0.(Some of the data i have in the set look like :

0.00123,0.4567655,0.0,0.212222,1 (Last value being the class, 1 to 4 the attributes, that 0.0 there would cause problems when i do P(Attr_1 | Yes) * P(Attr_2 | Yes) *.... ).

can you please help me out with this point as well?

I figured out what to do with the Test data/Training Data. I'm using a "10 fold" cross-validation technique.