Lecture 26 (01/06/2018): neural machine translation + end of the course!

The EM algorithm for word alignment in SMT. Beam search for decoding. Introduction to neural machine translation: the encoder-decoder neural architecture; back translation; byte pair encoding. The BLEU evaluation score. Performances and recent improvements. End of the course!

Lecture 25 (31/05/2018): bilingual/multilingual embeddings; semantic parsing

Bilingual and multilingual embeddings. Offline vs. online embeddings. Semantic parsing: definition, comparison to Semantic Role Labeling, approaches, a recent approach in detail. The Abstract Meaning Representation formalism. Introduction to machine translation (MT) and history of MT. Overview of statistical MT.

Lecture 24 (25/05/2018): semantic roles and semantic role labeling

From word to sentence representations. Semantic roles. Resources: PropBank, VerbNet, FrameNet. Semantic Role Labeling (SRL): traditional features. State-of-the-art neural approaches.

Lecture 23 (24/05/2018): issues in WSD; the knowledge acquisition bottleneck; sense distribution learning

Issues in Word Sense Disambiguation. Addressing the knowledge acquisition bottleneck for improving lexical semantic tasks. Learning sense distributions. 

Lecture 22 (18/05/2018): unsupervised and knowledge-based WSD

Unsupervised Word Sense Disambiguation: Word Sense Induction. Context-based clustering. Co-occurrence graphs: curvature clustering, HyperLex. Knowledge-based Word Sense Disambiguation. The Lesk and Extended Lesk algorithm. Structural approaches: similarity measures and graph algorithms. Conceptual density. Structural Semantic Interconnections. Evaluation: precision, recall, F1, accuracy. Baselines. Entity Linking.

Lecture 21 (17/05/2018): supervised Word Sense Disambiguation

Two important dimensions: supervision and knowledge. Supervised Word Sense Disambiguation: pros and cons. Vector representation of context. Main supervised disambiguation paradigms: decision trees, neural networks, instance-based learning, Support Vector Machines, IMS with embeddings, neural approaches to WSD.

Lecture 20 (11/05/2018): the attention mechanism with TensorFlow; introduction to Word Sense Disambiguation

Hands-on: the attention mechanism with TensorFlow. Introduction to Word Sense Disambiguation.

Lecture 19 (10/05/2018): semantic vector representations (2)

Semantic vector representations: importance of their multilinguality; linkage to BabelNet; latent vs. explicit representations; monolingual vs. multilingual representations. The NASARI lexical, unified and embedded representations.

Lecture 18 (04/05/2018): BabelNet; semantic vector representations (1)

Introduction to BabelNet (http://babelnet.org): multilingual synsets, resources integrated, accuracy, applications. Semantic vector representations: SensEmbed.

Lecture 17 (03/05/2018): computational lexicons; WordNet; introduction to WSD; homework 2

Encoding word senses: paper dictionaries, thesauri, machine-readable dictionary, computational lexicons. WordNet. Introduction to Word Sense Disambiguation (WSD). Homework 2: supervised and knowledge-based Word Sense Disambiguation.

Lecture 16 (27/04/2018): introduction to computational and lexical semantics

Introduction to computational semantics. Syntax-driven semantic analysis. Semantic attachments. First-Order Logic. Lambda notation and lambda calculus for semantic representation. Lexicon, lemmas and word forms. Word senses: monosemy vs. polysemy. Special kinds of polysemy. Computational sense representations: enumeration vs. generation. Graded word sense assignment.

Lecture 15 (26/04/2018): transition-based syntactic parser; introduction to semantics

Transition-based dependency parsing with buffer and stack; main transition rules and their implementation with BiLSTMs. Introduction to semantics.

Lecture 14 (20/04/2018): syntactic parsing (2/2)

The Early algorithm. Probabilistic CFGs. Probabilistic parsing, Neural transition-based dependency parsing with LSTMs: arc-factored dependency parsing.

Lecture 13 (19/4/2018): syntactic parsing (1/2)

Introduction to syntax. Context-free grammars and languages. Treebanks. Normal forms. Dependency grammars. Syntactic parsing: top-down and bottom-up. Structural ambiguity.  Backtracking vs. dynamic programming for parsing. The CKY algorithm.

Lecture 12 (13/04/2018): practical session on BiLSTMs with TensorFlow

Practical session on BiLSTMs with TensorFlow.

Lecture 11 (12/04/2018): character embeddings; multilingual POS tagging with word and character embeddings; computational syntax

Character embeddings; multilingual POS tagging with word and character embeddings; introduction to syntax. Context-free grammars and languages. Treebanks.

  

Lecture 10 (06/04/2018): Recurrent Neural Networks and LSTM; POS tagging with LSTMs in TensorFlow

Introduction to Recurrent Neural Networks (RNNs): definitions and configurations. Simple RNN, CBOW as RNN, gated architectures, Long-Short Term Memory networks (LSTMs). Neural POS tagging with LSTMs in TensorFlow.

Lecture 9 (05/04/2018): part-of-speech tagging

Stochastic part-of-speech tagging. Hidden markov models. Deleted interpolation. Linear and logistic regression: Maximum Entropy models. Transformation-based POS tagging. Handling out-of-vocabulary words. The Stanford POS tagger.

Lecture 8 (23/03/2018): more on word2vec; smoothing for language modeling; introduction to part-of-speech tagging

Use of word embeddings produced with word2vec. Smoothing techniques for probabilistic language modeling. Introduction to part-of-speech tagging: word classes; universal tag set.

Lecture 7 (22/03/2018): word2vec and its implementation

Word2vec: CBOW and skipgram; explanation, derivation of the loss function and implementation in TensorFlow.

Lecture 6 (16/03/2018): deep learning and word embeddings (1)

Introduction to neural networks. The perceptron. Neural units. Activation functions. MaxEnt and softmax. Word embeddings: rationale and word2vec. CBOW and skipgram. Homework 1 assignment!

Lecture 5 (15/03/2018): language modeling

We introduced N-gram models (unigrams, bigrams, trigrams), together with their probability modeling and issues. We discussed perplexity and its close relationship with entropy, we introduced smoothing

Lecture 4 (09/03/2018): TensorFlow for linear and polynomial regression; TensorBoard

More on TensorFlow: variables, placeholders, sessions, training. Linear and polynomial regression. TensorBoard.

Lecture 3 (08/03/2018): introduction to Machine Learning for NLP / TensorFlow

Introduction to Machine Learning for Natural Language Processing: supervised vs. unsupervised vs. reinforcement learning. Features, feature vector representations. TensorFlow.

Lecture 2 (02/03/2018): Introduction to NLP (2)

We continued our introduction to NLP, with a focus on the Turing Test as a tool to understand whether "machines can think". We also discussed the pitfalls of the test, including Searle's Chinese Room argument.

Lecture 1 (01/03/2018): Introduction to NLP (1)

We gave an introduction to the course and the field it is focused on, i.e., Natural Language Processing.

Ready, steady, go!

Welcome to the Sapienza NLP course blog! This year there will be important changes:

1. You will write a paper 
2. The course will be much more deep learning oriented
3. For attending students, there will be only three homeworks (and no additional duty), one of which will be done with delivery by the end of September and will replace the project. Non-attending students, instead, will have to work on a full-fledged project.

IMPORTANT: The 2018 class hour schedule will be on Thursday 16.30-19 and Fridays 14.00pm-16.30pm, Aula 2 - Aule L ingegneria.

Please sign up to the NLP class!