Course Syllabus & Policies
Comprehensive overview of Master's curriculum topics, learning outcomes, grading breakdown, prerequisites, and academic policies.
Course Description
Natural Language Processing (NLP) is at the epicenter of modern artificial intelligence. This graduate Master's course provides an in-depth mathematical and algorithmic foundation of modern NLP, focusing on large-scale language representation learning, Transformer architectures, Parameter-Efficient Fine-Tuning (PEFT/LoRA), Reinforcement Learning from Human Feedback (RLHF & DPO), Retrieval-Augmented Generation (RAG), and Multimodal LLMs.
Key Learning Outcomes
- Master mathematical foundations of autoregressive language models, word representations, and sequence modeling.
- Implement encoder-only, decoder-only, and encoder-decoder Transformer blocks from scratch with multi-head attention and RoPE.
- Understand pretraining scaling laws, compute-optimal recipes, data filtering, and distributed training systems.
- Apply parameter-efficient fine-tuning techniques (LoRA, QLoRA, prefix tuning) on open-weight LLMs.
- Implement preference alignment pipelines using RLHF, Direct Preference Optimization (DPO), and rejection sampling.
- Design industrial-grade Retrieval-Augmented Generation (RAG) pipelines with hybrid dense/sparse vector search.
- Analyze multimodal language models, decoding strategies, test-time scaling, and safety/fairness considerations.
Grading Policy & Breakdown
Official preliminary grading breakdown (110% Total with 10% bonus capacity)
| Component | Weight | Description | Visual Share |
|---|---|---|---|
| Assignment 1 | 10% | Word Representations, Autoregressive LMs & Tokenization | |
| Assignment 2 | 10% | Transformers & Parameter-Efficient Fine-Tuning (LoRA) | |
| Assignment 3 | 10% | RAG Systems, Reasoning (CoT) & Model Alignment (DPO) | |
| Assignment 4 (Course Project) | 20% | Course research and implementation project (Guidelines to be announced) | |
| Midterm Exam | 20% | Comprehensive exam covering Sessions S1 to S17 (Week 10) | |
| Final Exam | 20% | Final theoretical and applied examination on full semester topics | |
| Quizzes (Q1, Q2, Q3) | 15% | Three short review quizzes (5% each) at Weeks 5, 8, and 12 | |
| In-Class Participation | 5% | Active discussion, paper reviews, and classroom engagement | |
| Total Available Score | 110% | Includes 10% built-in bonus potential |
Curriculum Structure & Module Alignment
All 11 official curriculum modules are mapped independently to course sessions, aligning with national graduate requirements and benchmarked against CMU ANLP and Stanford CS224N.
Module 1: Introduction to Large Language Models & Distributed Semantics
History of NLP, distributed word representations (Word2Vec, GloVe), and autoregressive neural language modeling fundamentals.
Module 2: Pretraining Strategies, Data Scaling & Transfer Learning
Web scraping pipelines, quality filtering, deduplication, Chinchilla compute-optimal scaling laws, and transfer learning paradigms.
Module 3: Transformer Architecture & Subword Tokenization
Recurrent sequence encoders, scaled dot-product attention, multi-head Transformer blocks, RoPE, and subword tokenization (BPE, WordPiece).
Module 4: Security, Privacy, Hallucination & Robustness in NLP
Adversarial prompt injection, jailbreaking attacks, mechanistic interpretability, safety filters, and privacy preservation in LLMs.
Module 5: LLM Evaluation, Benchmarks & Leaderboards
Standardized NLP benchmarks (MMLU, GSM8K, HumanEval), LLM-as-a-Judge methodologies, Chatbot Arena Elo ranking, and contamination detection.
Module 6: Reinforcement Learning & Human Preference Alignment
Markov decision processes, PPO policy gradients, reward modeling, RLHF (InstructGPT), and Direct Preference Optimization (DPO).
Module 7: Advanced Architectures, Quantization & Parameter-Efficient Fine-Tuning
LoRA, QLoRA, weight quantization (AWQ, GPTQ), distributed Megatron/ZeRO training, and Mixture of Experts (MoE).
Module 8: Retrieval-Augmented Generation (RAG) & Dense Search
Dense passage retrieval (DPR, ColBERT), vector indexing (HNSW), chunking, re-ranking, and Self-RAG frameworks.
Module 9: Reasoning, Chain-of-Thought & Autonomous LLM Agents
Decoding algorithms, Chain-of-Thought prompting, test-time scaling/search, ReAct prompting, tool use, and autonomous agent loops.
Module 10: Multimodal Models & Diffusion Generation in NLP
Vision-Language Models (CLIP, LLaVA), cross-modal attention, continuous/discrete text diffusion, and flow matching for language generation.
Module 11: Graph Neural Networks (GNNs) for Natural Language Processing
Knowledge graph embeddings, Graph Convolutional Networks (GCN/GAT) for text classification, dependency parsing, and KG-LLM fusion.