Lecture 25 (27/05/2021, 3 hours): encoder-decoder architecture; multilingual Semantic Parsing; Neural Machine Translation

Introduction to multilingual Semantic Parsing. Abstract Meaning Represenatation. Introduction to machine translation (MT) and history of MT. Overview of statistical MT. Beam search for decoding. Introduction to neural machine translation: the encoder-decoder neural architecture. The BLEU evaluation score. Performances and recent improvements. Neural MT: the encoder-decoder architecture; BART; advantages; results. Attention in NMT. Additional uses of the encoder-decoder architecture: Generationary.

Closing of the course!

Lecture 24 (24/5/2021, 3 hours): more on WSD; Homework 3: WSD of Word-in-Context data; Semantic Role Labeling

More on Word Sense Disambiguation. Homework 3: WSD of Word-in-Context datasets. From word to sentence representations. Semantic roles. Resources: PropBank, VerbNet, FrameNet, VerbAtlas. Semantic Role Labeling (SRL). State-of-the-art neural approaches.

Lecture 23 (20/05/2021, 2.15 hours): Supervised and Knowledge-Based Word Sense Disambiguation

 Introduction to Word Sense Disambiguation. Elements necessary for performing WSD. Supervised vs. unsupervised vs. knowledge-based WSD. Supervised WSD techniques. Neural WSD: LSTM and Transformer-based approaches. Integration of knowledge and neural WSD.

Lecture 22 (17/05/2021, 2.15 hours): BERT, GLUE and SuperGLUE benchmarks, Word Sense Disambiguation

Byte Pair Encodings (BPEs). BERT. RoBERTa. XLM-R. Evaluation: GLUE and SuperGLUE benchmarks. 

Introduction to Natural Language Understanding (NLU): Word Sense Disambiguation, Semantic Role Labeling, Semantic Parsing. Lexical substitution. 

Lecture 21 (13/05/2021, 2.5 hours): attention and Transformers

Introduction to the attention in deep learning: motivation, attention scores, approaches. The Transformer architecture. Introduction to BERT.

Lecture 20 (10/05/2021, 3 hours): bilingual embeddings, contextualized word embeddings, ELMo + homework 2

More on semantic vector representations. Bilingual and multilingual embeddings. Contextualized word embeddings. ELMo. Presentation of homework 2: aspect-based sentiment analysis

Lecture 19 (06/05/2021, 2 hours): attention

 Introduction to attention (via the Question Answering task). PyTorch notebook.

Lecture 18 (03/05/2021, 2 hours): neural language modeling

 Introduction to Neural Language Modeling with LSTMs. Issues and advantages. PyTorch Notebook.

Lecture 17 (29/04/2021, 2 hours): semantic vector representations: SensEmbed and NASARI; linkage to BabelNet

Semantic vector representations: importance of their multilinguality; linkage to BabelNet; latent vs. explicit representations; monolingual vs. multilingual representations. The NASARI lexical, unified and embedded representations.

Lecture 16 (26/04/2021, 3 hours): intro to NER; semantic vector representations; Q&A on homework

Introduction to the Named Entity Recognition task. Semantic vector representations. Q&A on homework 1.

Lecture 15 (22/04/2021, 2 hours): POS tagging and NER with BiLSTMs

 Practical session: POS tagging and Named Entity Recognition with BiLSTMs.

Lecture 14 (19/04/2021, 3 hours): lexical semantics (2/2) and lexical-semantic knowledge resources

Human vs. computer dictionaries. Introduction to WordNet. The notion of synset. Lexical and semantic relations. Multilingual lexical-semantic knowledge graphs. BabelNet: motivation, creation, organization and evolution. What comes first? Language or concept? Two different views.

Lecture 13 (15/04/2021, 15-17, 2 hours): introduction to lexical semantics

Introduction to lexical semantics. Lexicon, lemmas and word forms. Word senses: monosemy vs. polysemy. Special kinds of polysemy. Computational sense representations: enumeration vs. generation. Graded word sense assignment.

Lecture 12 (12/04/2021, 14-17, 3 hours): jump into the future + RNN notebook

Talk on cross-lingual Semantic Role Labeling and Semantic Parsing (Cleopatra workshop at The Web Conference) + RNNs in PyTorch (notebook).

Lecture 11 (08/04/2021, 2.5 hours): Homework 1 presentation + RNNs practical session

Homework 1: the Word-in-Context disambiguation task. Recurrent Neural Networks in practice with PyTorch.

Lecture 10 (29/03/2021, 3.5 hours): more on word2vec, GloVe, RNNs, LSTMs and PyTorch Lightning

More on Word2Vec and word embeddings: hierarchical softmax; negative sampling. GloVe. Recurrent Neural Networks. Gated architectures, Long-Short Term Memory networks (LSTMs). Bidirectional LSTMs and stacked LSTMs. Character embeddings. Introduction to PyTorch Lightning

Lecture 9 (25/03/2021, 2 hours): part-of-speech tagging

Part-of-speech tagging. Hidden markov models. Deleted interpolation. Linear and logistic regression: Maximum Entropy models; logit and logistic function; relationship to sigmoid and softmax. Transformation-based POS tagging. Handling out-of-vocabulary words.