Lecture 1 (21/02/2022, 3 hours): introduction to NLP

We gave an introduction to the course and the field it is focused on, i.e., Natural Language Processing and its challenges. 

Ready, steady, go!

 Welcome to the Sapienza NLP course blog 2022! Cool things about to happen:

1. The course will contain lots of up-to-date content on deep learning, neural networks, and an improved hands-on with PyTorch and PyTorch Lightning!
2. For attending students, there will be only TWO homeworks (and no additional duty), one of which will be done with delivery by the end of September and will replace the project. Non-attending students, instead, will have to work on a three homeworks.
3. There will be cool challenges throughout the whole course, including the possibility of writing and publishing papers.

IMPORTANT: The current lecture model (TBC) is blended, meaning that 50% of the students can attend physically, while the others will attend via online streaming. Please get access to the Facebook group by signing up via the link below). For students who can attend physically, the 2022 class hour schedule will be on Monday 14-17 (room A5) and on Thursday 16-18 (room A3), via Ariosto, 25 (DIAG). Other students attending online will still be considered attending students.

Please sign up to the NLP class!

 

Lecture 25 (27/05/2021, 3 hours): encoder-decoder architecture; multilingual Semantic Parsing; Neural Machine Translation

Introduction to multilingual Semantic Parsing. Abstract Meaning Represenatation. Introduction to machine translation (MT) and history of MT. Overview of statistical MT. Beam search for decoding. Introduction to neural machine translation: the encoder-decoder neural architecture. The BLEU evaluation score. Performances and recent improvements. Neural MT: the encoder-decoder architecture; BART; advantages; results. Attention in NMT. Additional uses of the encoder-decoder architecture: Generationary.

Closing of the course!

Lecture 24 (24/5/2021, 3 hours): more on WSD; Homework 3: WSD of Word-in-Context data; Semantic Role Labeling

More on Word Sense Disambiguation. Homework 3: WSD of Word-in-Context datasets. From word to sentence representations. Semantic roles. Resources: PropBank, VerbNet, FrameNet, VerbAtlas. Semantic Role Labeling (SRL). State-of-the-art neural approaches.

Lecture 23 (20/05/2021, 2.15 hours): Supervised and Knowledge-Based Word Sense Disambiguation

 Introduction to Word Sense Disambiguation. Elements necessary for performing WSD. Supervised vs. unsupervised vs. knowledge-based WSD. Supervised WSD techniques. Neural WSD: LSTM and Transformer-based approaches. Integration of knowledge and neural WSD.

Lecture 22 (17/05/2021, 2.15 hours): BERT, GLUE and SuperGLUE benchmarks, Word Sense Disambiguation

Byte Pair Encodings (BPEs). BERT. RoBERTa. XLM-R. Evaluation: GLUE and SuperGLUE benchmarks. 

Introduction to Natural Language Understanding (NLU): Word Sense Disambiguation, Semantic Role Labeling, Semantic Parsing. Lexical substitution. 

Lecture 21 (13/05/2021, 2.5 hours): attention and Transformers

Introduction to the attention in deep learning: motivation, attention scores, approaches. The Transformer architecture. Introduction to BERT.

Lecture 20 (10/05/2021, 3 hours): bilingual embeddings, contextualized word embeddings, ELMo + homework 2

More on semantic vector representations. Bilingual and multilingual embeddings. Contextualized word embeddings. ELMo. Presentation of homework 2: aspect-based sentiment analysis

Lecture 19 (06/05/2021, 2 hours): attention

 Introduction to attention (via the Question Answering task). PyTorch notebook.

Lecture 18 (03/05/2021, 2 hours): neural language modeling

 Introduction to Neural Language Modeling with LSTMs. Issues and advantages. PyTorch Notebook.