Lecture 8 (23/03/2018): more on word2vec; smoothing for language modeling; introduction to part-of-speech tagging

Use of word embeddings produced with word2vec. Smoothing techniques for probabilistic language modeling. Introduction to part-of-speech tagging: word classes; universal tag set.

Lecture 7 (22/03/2018): word2vec and its implementation

Word2vec: CBOW and skipgram; explanation, derivation of the loss function and implementation in TensorFlow.

Lecture 6 (16/03/2018): deep learning and word embeddings (1)

Introduction to neural networks. The perceptron. Neural units. Activation functions. MaxEnt and softmax. Word embeddings: rationale and word2vec. CBOW and skipgram. Homework 1 assignment!

Lecture 5 (15/03/2018): language modeling

We introduced N-gram models (unigrams, bigrams, trigrams), together with their probability modeling and issues. We discussed perplexity and its close relationship with entropy, we introduced smoothing

Lecture 4 (09/03/2018): TensorFlow for linear and polynomial regression; TensorBoard

More on TensorFlow: variables, placeholders, sessions, training. Linear and polynomial regression. TensorBoard.

Lecture 3 (08/03/2018): introduction to Machine Learning for NLP / TensorFlow

Introduction to Machine Learning for Natural Language Processing: supervised vs. unsupervised vs. reinforcement learning. Features, feature vector representations. TensorFlow.

Lecture 2 (02/03/2018): Introduction to NLP (2)

We continued our introduction to NLP, with a focus on the Turing Test as a tool to understand whether "machines can think". We also discussed the pitfalls of the test, including Searle's Chinese Room argument.

Lecture 1 (01/03/2018): Introduction to NLP (1)

We gave an introduction to the course and the field it is focused on, i.e., Natural Language Processing.

Ready, steady, go!

Welcome to the Sapienza NLP course blog! This year there will be important changes:

1. You will write a paper 
2. The course will be much more deep learning oriented
3. For attending students, there will be only three homeworks (and no additional duty), one of which will be done with delivery by the end of September and will replace the project. Non-attending students, instead, will have to work on a full-fledged project.

IMPORTANT: The 2018 class hour schedule will be on Thursday 16.30-19 and Fridays 14.00pm-16.30pm, Aula 2 - Aule L ingegneria.

Please sign up to the NLP class!

Lecture 13: semantic parsing; statistical and neural machine translation

Semantic parsing: supervised, unsupervised, semi-supervised. FrameNet. Abstract Meaning Representations.

Introduction to Machine Translation. Rule-based vs. Statistical MT. Statistical MT: the noisy channel model. The language model and the translation model. The phrase-based translation model. Learning a model of training. Phrase-translation tables. Parallel corpora. Extracting phrases from word alignments. Word alignments. IBM models for word alignment. Many-to-one and many-to-many alignments. IBM model 1 and the HMM alignment model. Training the alignment models: the Expectation Maximization (EM) algorithm. Symmetrizing alignments for phrase-based MT: symmetrizing by intersection; the growing heuristic. Calculating the phrase translation table. Decoding: stack decoding. Evaluation of MT systems. BLEU. Neural MT: the encoder-decoder architecture; advantages; results.