This course introduces the fundamental concepts underlying Large Language Models (LLMs). It starts with an introduction to the various problems in NLP, and discusses how to approach the problem of language modeling using deep learning. It describes the architectural intricacies of Transformers and the pre-training objectives of the differen Transformer-based models. It also discusses the recent advances in LLM research, including LLM alignment, prompting, parameter-efficient adaptation, hallucination, bias and ethical considerations. This course prepares a student to comprehend, critique and approach various research problems on LLMs.
Aulas do Curso
(38 vídeos)
0 de 38 aulas concluídas
0%
Progresso salvo somente neste navegador.
0
Introduction to Large Language Models (LLMs)
Dive deep into the world of Large Language Models (LLMs) with this cutting-edge course by IIT Delhi & IIT Bombay! Whethe…
1
Lec 01 | Introduction and Recent Advances
This video offers an introduction to the course's content, logistics, policies, and background, providing a solid founda…
2
Lec 02 | Introduction to Natural Language Processing
This video offers a detailed overview of the key terminologies and challenges in NLP, serving as a perfect gateway for t…
3
Lec 03 | Introduction to Statistical Language Models
This video provides an introduction to statistical language modeling, covering the essential methodologies and their app…
4
Lec 04 | Statistical LM: Advanced Smoothing and Evaluation
This video delves into advanced techniques for smoothing and both intrinsic and extrinsic methods for evaluating languag…
5
Lec 05 | Introduction to Deep Learning
This video introduces the fundamentals of deep learning, covering perceptrons, artificial neural networks (ANN), and bac…
6
Lec 06 | Introduction to PyTorch
In the tutorial, we cover the fundamentals of PyTorch - starting with its key components namely tensors and Autograd. Ne…
7
Lec 07 | Word Representation: Word2Vec & fastText
This lecture covers essential techniques for representing words as vectors, from traditional count-based methods to adva…
8
Lec 08 | Word Representation: GloVe
This video presentation explores the GloVe model's innovative methodology that effectively combines count-based and pred…
9
Lec 09 | Tokenization Strategies
This lecture covers key tokenization strategies such as Byte-Pair Encoding, WordPiece, and Unigram Language Model, essen…
10
Lec 10 | Neural Language Models: CNN & RNN
This lecture provides an overview of language modeling using neural network variants like CNNs and RNNs, focusing on the…
11
Lec 11 | Neural Language Models: LSTM & GRU
This lecture addresses the inherent problems with RNNs, particularly vanishing gradients, and explores the architectural…
12
Lec 12 | Sequence-to-Sequence Models
This lecture covers sequence-to-sequence modeling, explaining the encoder-decoder architecture, training, and decoding s…
13
Lec 13 | Decoding Strategies
This lecture explores decoding strategies in sequence-to-sequence models, covering deterministic methods like greedy dec…
14
Lec 14 | Attention in Sequence-to-Sequence Models
tl;dr: This lecture explores the limitations of RNN-based sequence-to-sequence models, particularly the bottleneck probl…
15
Lec 15 | Introduction to Transformer: Self & Multi-Head Attention
This lecture introduces the Transformer model, explaining its groundbreaking approach to language modeling and sequence …
16
Lec 16 | Introduction to Transformer: Positional Encoding and Layer Normalization
This lecture dives into the technical aspects of positional encoding methods and layer normalization within the Transfor…
17
Lec 17 | Implementation of Transformer using PyTorch
tl;dr: In this tutorial, we cover the implementation of a Transformer model in PyTorch, starting with the self-attention…
18
Lec 18 | Pre-Training Strategies: ELMo, BERT
tl;dr: This lecture explores the critical pre-training strategies behind influential models like ELMo and BERT, emphasiz…
19
Lec 19 | Pre-Training Strategies: Encoder-decoder and Decoder-only Models
tl;dr: This lecture explores pre-training strategies for encoder-decoder (T5, BART) and decoder-only (GPT, LLaMA) models…
20
Lec 20 | Introduction to HuggingFace
tl;dr: This tutorial introduces HuggingFace’s libraries for NLP tasks. It covers tokenization, model loading, inference,…
21
Lec 21 | Instruction Tuning
tl;dr: This lecture discusses instruction tuning as a method to improve LLMs' ability to generalize to unseen tasks by f…
22
Lec 22 | Prompt-based Learning
tl;dr: This lecture discusses prompt-based learning as an alternative to fine-tuning LLMs, exploring how different promp…
23
Lec 23 | Advanced Prompting and Prompt Sensitivity
tl;dr: This lecture discusses advanced prompting techniques like chain-of thought, tree-of-thought and graph-of-thought …
24
Lec 24 | Alignment of Language Models-I
tl;dr: This lecture discusses aligning LLMs through reinforcement learning and reward modeling, focusing on RLHF and the…
25
Lec 25 | Alignment of Language Models-II
tl;dr: This lecture discusses optimizing LLM alignment using policy gradient methods like REINFORCE and PPO, focusing on…
26
Lec 26 | Knowledge and Retrieval: Knowledge Graph
tl;dr; This lecture explores knowledge graphs and their retrieval from large language models. It also covers key concept…
27
Lec 27 | Knowledge and Retrieval: Knowledge Graph Completion and Evaluation
tl;dr; In this lecture, we will learn in detail about knowledge graph completion, understanding the loss function along …
28
Lec 28 | Knowledge and Retrieval: Translation and Rotation Models
tl;dr; In this lecture, we will be learning in details about various graph embedding approaches such as TransE, TransH a…
29
Lec 29 | Parameter Efficient Fine-Tuning (PEFT)
tl;dr: This lecture covers various techniques of Parameter Efficient Fine-Tuning (PEFT) that enable significant modifica…
30
Lec 30 | Quantization, Pruning & Distillation
tl;dr: This lecture covers various effective model compression techniques such as quantization, pruning, and distillatio…
31
Lec 31 | An Alternate Formulation of Transformers: Residual Stream Perspective
tl;dr: This lecture delves into the residual stream perspective of Transformers, providing a detailed breakdown of how e…
32
Lec 32 | Interpretability Techniques
tl;dr: This lecture covers a range of interpretability techniques that aim to shed light on the internal mechanisms of L…
33
Lec 33 | Knowledge and Retrieval: Multiplicative models
tl;dr: This lecture covers a family of knowledge graph embeddings based on multiplicative or factor models. We will star…
34
Lec 34 | Knowledge and Retrieval: Modeling Hierarchies
tl;dr: Transitive relations play a key role in applications like information extraction and question answering. Understa…
35
Lec 35 | Knowledge and Retrieval: Temporal Knowledge Graphs
tl;dr: Knowledge graph representations discussed in earlier lectures did not account for time. This lecture covers diffe…
36
Lec 36 | Responsible LLMs
tl;dr: This lecture discusses the importance of building responsible LLMs by addressing issues like bias, fairness, and …
37
Lec 37 | Conclusion: Expert Panel Discussion
tl;dr: This video brings together leading experts to discuss current trends in large language models (LLMs), their appli…
