Lecture 24 (24/5/2021, 3 hours): more on WSD; Homework 3: WSD of Word-in-Context data; Semantic Role Labeling

More on Word Sense Disambiguation. Homework 3: WSD of Word-in-Context datasets. From word to sentence representations. Semantic roles. Resources: PropBank, VerbNet, FrameNet, VerbAtlas. Semantic Role Labeling (SRL). State-of-the-art neural approaches.

Lecture 23 (20/05/2021, 2.15 hours): Supervised and Knowledge-Based Word Sense Disambiguation

 Introduction to Word Sense Disambiguation. Elements necessary for performing WSD. Supervised vs. unsupervised vs. knowledge-based WSD. Supervised WSD techniques. Neural WSD: LSTM and Transformer-based approaches. Integration of knowledge and neural WSD.

Lecture 22 (17/05/2021, 2.15 hours): BERT, GLUE and SuperGLUE benchmarks, Word Sense Disambiguation

Byte Pair Encodings (BPEs). BERT. RoBERTa. XLM-R. Evaluation: GLUE and SuperGLUE benchmarks. 

Introduction to Natural Language Understanding (NLU): Word Sense Disambiguation, Semantic Role Labeling, Semantic Parsing. Lexical substitution. 

Lecture 21 (13/05/2021, 2.5 hours): attention and Transformers

Introduction to the attention in deep learning: motivation, attention scores, approaches. The Transformer architecture. Introduction to BERT.

Lecture 20 (10/05/2021, 3 hours): bilingual embeddings, contextualized word embeddings, ELMo + homework 2

More on semantic vector representations. Bilingual and multilingual embeddings. Contextualized word embeddings. ELMo. Presentation of homework 2: aspect-based sentiment analysis

Lecture 19 (06/05/2021, 2 hours): attention

 Introduction to attention (via the Question Answering task). PyTorch notebook.

Lecture 18 (03/05/2021, 2 hours): neural language modeling

 Introduction to Neural Language Modeling with LSTMs. Issues and advantages. PyTorch Notebook.