Anyone Recommend a Good Tutorial on Conditional Random Fields

Question

I have been trying to find a good tutorial on Conditional Random Fields and have yet to find one that didn't start sending my brain into meltdown. I have a good grasp of HMM's, and I get the difference between discriminative and generative models ... but as of yet I have not been able to find a resource that can give a good comparison of HMM's and CRF's that makes sense to me. Any assistance would be appreciated.

Answer 1

One of the best resources I've found is actually a section in Christopher Bishop's book Pattern Recognition and Machine Learning (which I highly recommend, by the way) regarding Markov Random Fields (CRFs are specialized Markov Random Fields.) It even has an example, which I'm sure you've noticed by now are incredibly hard to find for this subject. Now, I must stipulate that this section won't give you a complete understanding of CRFs, but it will hopefully - at least it did for me - help you navigate those treacherous CRF tutorials.

Aside from that, I haven't found anything but mind-numbing academic papers on the subject. Here are a few that I found to be helpful though:

• An Introduction to Conditional Random Fields for Relational Learning - This one is quite thorough.

• Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data - This is the paper which originally outlined the CRF framework. I find it a bit easier to read than the ones that came after it. It also compares CRFs to HMMs and MEMMs (diagrams included.)

Sorry, that's all I can contribute. I'm still trying to master CRFs myself.

Answer 2

Classical Probabilistic Models and Conditional Random Fields

http://www.scai.fraunhofer.de/fileadmin/images/bio/data_mining/paper/crf_klinger_tomanek.pdf

This is by far the best tutorial I've so far had the good fortune to come across. As the title implies, it develops the CRM idea by first building on top of and relating to more familiar models including Naive Bayes, HMMs, and Maximum Entropy. Use of colors and figures also adds to understandability.

Answer 3

A very interesting video tutorial on CRFs given by Prof. Charles Elkan (UCSD): http://videolectures.net/cikm08%5Felkan%5Fllmacrf

And lecture notes can be downloaded from his homepage: cseweb.ucsd.edu/users/elkan/250B/cikmtutorial.pdf

Cheers! Hung Ngo.

Answer 4

This classroom handout explains the "notation" for a Linear-chain CRF rather well. And is perhaps the most gentle, hand-held introduction to CRF that I have read so far.