Introduction to Neural Models for AI

• 26 hrs lectures
• 13 hrs seminar
• 12 hrs projects

• 52 pts final exam
• 16 pts mid-term test
• 8 pts written tests
• 24 pts projects

Allow students to understand the fundamental building blocks of current generation AI models that are based on large language models and chat bots. The course builds theoretical concepts with practical sessions (jupyter notebooks, labs) incrementally leading to final tangible projects.

Basic of Programming (Python, C).

Fundamentals of Probability Theory, Statistics, and Mathematical Analysis.

The course also covers basic concepts from the field of probability theory, statistics, and mathematical analysis, but some prior knowledge in these areas will be an advantage.

• Machine Learning with PyTorch and Scikit-Learn. Packt Publishing Ltd. ISBN-10: 1801819319 ISBN-13: 978-1801819312. https://github.com/rasbt/machine-learning-book
• Natural Language Processing with Transformers. O'Reilly https://github.com/nlp-with-transformers

• Deep Learning - Foundations and Concepts. Springer https://link.springer.com/book/10.1007/978-3-031-45468-4
• Deep learning. MIT Press. https://www.deeplearningbook.org/

1. Introduction to present-day AI
2. Perceptron model 
3. Word embeddings: purpose, training, interpretation
4. Multi-layer perceptron (MLPs)
5. Feedforward NN for Language Modelling
6. Optimization in MLPs
7. Recurrent NN for LM
8. Transformer LM
9. Fine-tuning pre-trained LMs
10. Instruction tuning and Chain of thought
11. Fine-tuning with human preferences
12. Reinforcement learning - Agentic systems
13. Poster session - project demonstrations (depending on number of students) or Guest lecture

1. The final exam will be in written form if there are too many students. Oral exam if the number of students is around 30. (52 points)
2. Project (24 points) - evaluation method will be decided based on the number of students and groups.
3. A couple of the class tests will be considered as mid-terms. These will be 30 minute long multiple-choice-question (MCQ) based exams with penalties (~30 min). (16 points).
4. Non-midterm class tests (MCQ format) will contribute to half the points as mid-term because they will be shorter in duration (~15 min) (8 points).