Lecture 6: deep learning; intro to part of speech tagging

Recurrent Neural Networks and Long-Short Term Memory networks. Practical session on character-based LSTMs with Keras. Introduction to part-of-speech tagging.

Lecture 5: practical session on Keras; more on NNs for NLP; word embeddings

Practical session on Keras. More on NNs for NLP: hierarchical softmax; negative sampling. Vector representations. Word2vec. Word embeddings and their properties.

Lecture 4: language modeling (2); neural networks and NLP

We discussed perplexity and its close relationship with entropy, we introduced smoothing and interpolation techniques to deal with the issue of data sparsity. Practical session on language modeling with Python and the Berkeley LM toolkit.

Lecture 3: morphological analysis: practical session; homework 1; language modeling (1)

We had a practical session on morphological analysis in Python and Java. We reviewed basic probability concepts. introduced N-gram models (unigrams, bigrams, trigrams), together with their probability modeling and issues.

We also discussed homework 1 (see post on the class group).

Lecture 2: intro (2); morphological analysis

We introduced words and morphemes. Before delving into morphology and morphological analysis, we introduced regular expressions as a powerful tool to deal with different forms of a word. We then introduced recent work on morphological analysis based on machine learning: unsupervised (Morfessor) and supervised (based on CRFs).