Lecture 3: language modeling

We introduced N-gram models (unigrams, bigrams, trigrams), together with their probability modeling and issues. We discussed perplexity and its close relationship with entropy, we introduced smoothing and interpolation techniques to deal with the issue of data sparsity. The KYOTO and Berkley Language Model toolkits.

We also discussed the homework 1 in more detail (see slides on the class group).