Understanding Bayes' Theorem

Question

I'm working on an implementation of A Naive Bayes Classifier. Programming Collective Intelligence introduces this subject by describing Bayes Theorem as:

Pr(A | B) = Pr(B | A) x Pr(A)/Pr(B)

As well as a specific example relevant to document classification:

Pr(Category | Document) = Pr(Document | Category) x Pr(Category) / Pr(Document)

I was hoping someone could explain to me the notation used here, what do Pr(A | B) and Pr(A) mean? It looks like some sort of function but then what does the pipe mean, etc? (I am a little lost)

Thanks in advance.

Answer 1

• Pr(A | B) = Probability of A happening given that B has already happened
• Pr(A) = Probability of A happening

But the above is with respect to the calculation of conditional probability. What you want is a classifier, which uses this principle to decide whether something belongs to a category based on the previous probability.

See http://en.wikipedia.org/wiki/Naive%5FBayes%5Fclassifier for a complete example

Answer 2

Pr is the probability, Pr(A|B) is the conditional probability.

Check wikipedia for details.

Answer 3

Pr(A | B): Conditional probability of A : i.e. probability of A, given that all we know is B

Pr(A) : Prior probability of A

Answer 4

the pipe (|) means "given". The probability of A given B is equal to the probability of B given A x Pr(A)/Pr(B)

Answer 5

Based on your question I can strongly advise that you need to read some undergraduate book on Probability Theory first. Without this you will not advance properly with your task on Naive Bayes Classifier.

I would recommend you this book http://www.athenasc.com/probbook.html or look at MIT OpenCourseWare.