Assignments & Project Work
Structured hands-on assignments (HW1–HW3) and capstone research project to implement, train, and evaluate state-of-the-art NLP models.
Word Representations, Neural LMs & Tokenization
Implement Skip-gram with negative sampling, build an autoregressive character/word level recurrent language model, and implement Byte Pair Encoding (BPE) from scratch.
Expected Deliverables:
- Jupyter Notebook with code & results
- Written report (PDF)
- Empirical analysis on embeddings
The Transformer Architecture & Parameter-Efficient Fine-Tuning (LoRA)
Construct a multi-head decoder-only Transformer block from scratch in PyTorch, implement Rotary Positional Embeddings (RoPE), and fine-tune an open-source LLM using LoRA on instruction datasets.
Expected Deliverables:
- PyTorch code package
- Evaluation curves & ablation table
- Submission to leaderboard
RAG Systems, Reasoning (CoT) & Model Alignment (DPO)
Build a hybrid dense-sparse vector RAG pipeline for knowledge retrieval, implement test-time search/reasoning verification, and perform preference alignment using Direct Preference Optimization (DPO).
Expected Deliverables:
- Complete RAG application script
- DPO loss implementation
- Comparison metrics report
Course Research & Engineering Project
Details, research scope, proposal instructions, and milestone timelines for the course project will be announced by the instructor during the semester.
Expected Deliverables:
- Project Proposal (TBA)
- Progress Milestone (TBA)
- Project Presentation (Week 15)
- Final Code & Technical Report (TBA)