BSDA5004 · workspace

Llms

Syllabus, study tools, lectures, and curriculum map.

← Back to hub

More courses

Browse course shelf

Weekly outline

Syllabus

Week topics from the course map

00W00

Topic

Incomplete

01W01

Transformers: Introduction to transformers - Self-attention - cross- attention-Masked attention-Positional encoding

Incomplete

02W02

A deep dive into number of parameters, computational complexity and FLOPs- Introduction to language modeling

Incomplete

03W03

Causal Language Modeling: What is a language model?- Generative Pretrained Transformers (GPT) - Training and inference

Incomplete

04W04

Masked Language Modeling : Bidirectional Encoder Representations of Transformers (BERT) - Fine-tuning - A deep dive into tokenization: BPE, SentencePi

Incomplete

05W05

Bigger Picture: T5, A deep dive into text-to-text (genesis of prompting), taxonomy of models, road ahead

Incomplete

06W06

Data: Datasets, Pipelines, effectiveness of clean data, Architecture: Types of attention, positional encoding (PE) techniques, scaling techniques

Incomplete

07W07

Training: Revisiting optimizers, LION vs Adam, Loss functions, Learning schedules, Gradient Clipping, typical failures during training

Incomplete

08W08

Fine Tuning: Prompt Tuning,Multi-task Fine-tuning,Parametric Efficient

Incomplete

09W09

Benchmarks: MMLU, BigBench, HELM,OpenLLM, Evaluation

Incomplete

010W10

Training Large Models: Mixed precision training,Activation checkpointing, 3D parallelism, ZERO, Bloom as a case study

Incomplete

011W11

Scaling Laws: Chinchilla,Gopher, Palm v2

Incomplete

012W12

Recent advances

Incomplete

Study tools

Memorize first

Compact recall strip

Open notes

6 min

BERT Architecture: Encoder-Only & Bidirectional Context

Open this first for the core logic and traps.

Dive in

9 min

Fine-tuning Methods: LoRA, Adapters, and PEFT

Open this first for the core logic and traps.

Dive in

9 min

Layer Normalization & Residual Connections in Transformers

Open this first for the core logic and traps.

Dive in

Practice Dock

One drill at a time, with Theo hints, teacher lanes, and a replay queue.

Use this when the theory is already clear and the goal is fast, clean retrieval under pressure. Mark weak questions, copy a revision pack, and move on.

TheoTeacherTextbookSpeedrun

Ask less. Recall more.

Open Practice Dock

SM-2 Active

Spaced Repetition

Flashcards

Spaced repetition engine for long-term retention of core concepts.

Start session

Not Available

Interactive Simulations

Visual Labs

Exam Mode

Adaptive Practice

Mock Exam

Set paper

Code Protocol

OPPE Simulator

Secure Comm-Link Terminal

Secure Comm-Link // @IITMadrasBSDegreeProgramme

Uplink 12ms

Syllabus Synchronization: Active

Open current cue

Syllabus Matrix Registry

Global Course Index

Open Full Frame

Master Registry

v6.4 Directory

Foundational

Diploma

BSc Degree

BS Degree

PG / MTech

BSDA5004

PG / MTech

4 Credits

Large Language Models

Understanding the Transformer architecture Understanding the concept of pretraining and fine-tuning language models Compare and contrast different ...

Required Dependencies:

Deployment Hub External Docs

Execution Protocol

Module 0

Topic

Module 1

Transformers: Introduction to transformers - Self-attention - cross- attention-Masked attention-Positional encoding

Module 2

A deep dive into number of parameters, computational complexity and FLOPs- Introduction to language modeling

Module 3

Causal Language Modeling: What is a language model?- Generative Pretrained Transformers (GPT) - Training and inference

Module 4

Masked Language Modeling : Bidirectional Encoder Representations of Transformers (BERT) - Fine-tuning - A deep dive into tokenization: BPE, SentencePi

Module 5

Bigger Picture: T5, A deep dive into text-to-text (genesis of prompting), taxonomy of models, road ahead

Module 6

Data: Datasets, Pipelines, effectiveness of clean data, Architecture: Types of attention, positional encoding (PE) techniques, scaling techniques

Module 7

Training: Revisiting optimizers, LION vs Adam, Loss functions, Learning schedules, Gradient Clipping, typical failures during training

Module 8

Fine Tuning: Prompt Tuning,Multi-task Fine-tuning,Parametric Efficient

Module 9

Benchmarks: MMLU, BigBench, HELM,OpenLLM, Evaluation

Module 10

Syllabus

Topic

Transformers: Introduction to transformers - Self-attention - cross- attention-Masked attention-Positional encoding

A deep dive into number of parameters, computational complexity and FLOPs- Introduction to language modeling

Causal Language Modeling: What is a language model?- Generative Pretrained Transformers (GPT) - Training and inference

Masked Language Modeling : Bidirectional Encoder Representations of Transformers (BERT) - Fine-tuning - A deep dive into tokenization: BPE, SentencePi

Bigger Picture: T5, A deep dive into text-to-text (genesis of prompting), taxonomy of models, road ahead

Data: Datasets, Pipelines, effectiveness of clean data, Architecture: Types of attention, positional encoding (PE) techniques, scaling techniques

Training: Revisiting optimizers, LION vs Adam, Loss functions, Learning schedules, Gradient Clipping, typical failures during training

Fine Tuning: Prompt Tuning,Multi-task Fine-tuning,Parametric Efficient

Benchmarks: MMLU, BigBench, HELM,OpenLLM, Evaluation

Training Large Models: Mixed precision training,Activation checkpointing, 3D parallelism, ZERO, Bloom as a case study

Scaling Laws: Chinchilla,Gopher, Palm v2

Recent advances

Compact recall strip

One drill at a time, with Theo hints, teacher lanes, and a replay queue.

Flashcards

Visual Labs

Mock Exam

OPPE Simulator

Secure Comm-Link Terminal

Syllabus Matrix Registry

Master Registry

Foundational

Diploma

BSc Degree

BS Degree

PG / MTech

Large Language Models

Execution Protocol

Topic

Transformers: Introduction to transformers - Self-attention - cross- attention-Masked attention-Positional encoding

A deep dive into number of parameters, computational complexity and FLOPs- Introduction to language modeling

Causal Language Modeling: What is a language model?- Generative Pretrained Transformers (GPT) - Training and inference

Masked Language Modeling : Bidirectional Encoder Representations of Transformers (BERT) - Fine-tuning - A deep dive into tokenization: BPE, SentencePi

Bigger Picture: T5, A deep dive into text-to-text (genesis of prompting), taxonomy of models, road ahead

Data: Datasets, Pipelines, effectiveness of clean data, Architecture: Types of attention, positional encoding (PE) techniques, scaling techniques

Training: Revisiting optimizers, LION vs Adam, Loss functions, Learning schedules, Gradient Clipping, typical failures during training

Fine Tuning: Prompt Tuning,Multi-task Fine-tuning,Parametric Efficient

Benchmarks: MMLU, BigBench, HELM,OpenLLM, Evaluation

Training Large Models: Mixed precision training,Activation checkpointing, 3D parallelism, ZERO, Bloom as a case study

Scaling Laws: Chinchilla,Gopher, Palm v2

Recent advances

Video Archive

Intelligence Hub