Introduction to machine translation (MT) and history of MT. Overview of statistical MT. Beam search for decoding. Introduction to neural machine translation: the encoder-decoder
neural architecture. The BLEU
evaluation score. Performances and recent improvements. Neural MT: the encoder-decoder architecture; Attention in NMT. Closing of the course!
Semantic Parsing, AMR and BMR (BabelNet Meaning Representation), Natural Language Generation and stochastic parrots.
Neural WSD: Transformer-based approaches. Integration of knowledge and neural WSD. Datasets and issues in WSD. Scaling multilingually. Introduction to coreference resolution. Presentation of homework 3.
Introduction to Word Sense Disambiguation: introduction to the task. Purely data-driven, and neuro-symbolic approaches. Issues. Semantic Role Labeling: introduction to the task. Inventories. Neural approaches. Issues. Homework 2 assignment.
The Transformer architecture. Rationale. Self- and cross-attention; keys, queries, values. Encoder and decoder blocks.
Pre-training and fine-tuning. Encoder, decoder and encoder-decoder pre-trained models. GPT, BERT. Masked language modeling and next-sentence prediction tasks. Practical session on the Transformer with BERT.
