Course Syllabus & Policies
Comprehensive overview of Master's curriculum topics, learning outcomes, grading breakdown, prerequisites, and academic policies.
Course Description
Natural Language Processing (NLP) is at the epicenter of modern artificial intelligence. This graduate Master's course provides an in-depth mathematical and algorithmic foundation of modern NLP, focusing on large-scale language representation learning, Transformer architectures, Parameter-Efficient Fine-Tuning (PEFT/LoRA), Reinforcement Learning from Human Feedback (RLHF & DPO), Retrieval-Augmented Generation (RAG), and Multimodal LLMs.
Key Learning Outcomes
- Master mathematical foundations of autoregressive language models, word representations, and sequence modeling.
- Implement encoder-only, decoder-only, and encoder-decoder Transformer blocks from scratch with multi-head attention and RoPE.
- Understand pretraining scaling laws, compute-optimal recipes, data filtering, and distributed training systems.
- Apply parameter-efficient fine-tuning techniques (LoRA, QLoRA, prefix tuning) on open-weight LLMs.
- Implement preference alignment pipelines using RLHF, Direct Preference Optimization (DPO), and rejection sampling.
- Design industrial-grade Retrieval-Augmented Generation (RAG) pipelines with hybrid dense/sparse vector search.
- Build autonomous LLM agent loops using function calling, tool use, and ReAct decision frameworks.
- Analyze multimodal language models, decoding strategies, test-time scaling, and safety/fairness considerations.
Grading Policy & Breakdown
Official course grading breakdown (100% Total Scale)
| Component | Weight | Description | Visual Share |
|---|---|---|---|
| Multi-Phase Course Project | 45% | Hands-on phased capstone project released in stages across the semester (Milestones, Codebase & Final Evaluation) | |
| Quizzes (Q1, Q2, Q3) | 20% | Three progress review quizzes (totaling 20%): Q1 (Week 4, covers S1–S6), Q2 (Week 7, covers S7–S12), and Q3 (Week 11, covers S13–S18) | |
| Final Exam | 30% | Comprehensive examination covering theoretical and applied topics from Sessions S13 to S25 (Wednesday, 1405-10-30 • 11:00–13:00) | |
| In-Class Participation | 5% | Active discussion, paper reviews, and classroom engagement | |
| Total Available Score | 100% | Standard 100% evaluation basis |
Curriculum Structure & Module Alignment
All 11 official curriculum modules are mapped systematically to course sessions, aligning with national graduate requirements and benchmarked against CMU ANLP and Stanford CS224N.
Module 1: Introduction to Large Language Models & Distributed Semantics
History of NLP, distributed word representations (Word2Vec, GloVe), neural network foundations, and autoregressive language modeling.
Module 2: Pretraining Strategies, Data Scaling & Transfer Learning
Web scraping data pipelines, quality filtering, deduplication, and Chinchilla compute-optimal scaling laws.
Module 3: Transformer Architecture, Sequential Encoders & Subword Tokenization
Recurrent sequence encoders (RNN/LSTM/GRU), scaled dot-product attention, multi-head Transformer blocks, RoPE, and subword tokenization (BPE, SentencePiece).
Module 4: Security, Privacy, Hallucination & Robustness in NLP
Adversarial prompt injection, jailbreaking attacks, mechanistic interpretability, safety filters, and privacy preservation in LLMs.
Module 5: LLM Evaluation, Benchmarks & Leaderboards
Standardized NLP benchmarks (MMLU, GSM8K), LLM-as-a-Judge methodologies, Chatbot Arena Elo ranking, and contamination detection.
Module 6: Reinforcement Learning & Human Preference Alignment
Markov decision processes, PPO policy gradients, reward modeling, RLHF (InstructGPT), and Direct Preference Optimization (DPO).
Module 7: Advanced Architectures, Quantization & Parameter-Efficient Fine-Tuning
LoRA, QLoRA, weight quantization (AWQ, GPTQ), FlashAttention-2, and Mixture of Experts (MoE).
Module 8: Retrieval-Augmented Generation (RAG) & Dense Search
Dense passage retrieval (DPR, ColBERT), vector indexing (HNSW), chunking, re-ranking, and Self-RAG frameworks.
Module 9: Reasoning, Chain-of-Thought, Test-Time Scaling & Autonomous LLM Agents
Decoding algorithms, Chain-of-Thought prompting, test-time compute scaling, process supervision (PRM), ReAct prompting, tool use, and autonomous agent loops.
Module 10: Multimodal Models & Diffusion Generation in NLP
Vision-Language Models (CLIP, LLaVA), cross-modal attention, continuous/discrete text diffusion, and flow matching for language generation.
Module 11: Graph Neural Networks (GNNs) & Research Frontiers in NLP
Knowledge graph embeddings, Graph Convolutional Networks (GCN/GAT) for NLP, ACL experimental design, and open research frontiers.