Laboratory of Computational Neuroscience LCN - Brain Mind - I&C - Life Sciences - EPFL

EPFL Graduate Course (Master program):

Unsupervised and Reinforcement Learning in Neural Networks

• Unsupervised and Reinforcement Learning in Neural Networks (Fall term; 2h of lectures and 2h of exercises per week)
 

This course for Computer Scientists and Life Scientists focuses on biological modeling of the process of learning in neural systems. In contrast to the course on `Pattern classification and machine learning' which focuses on algorithmic approaches towards supervised learning, this course covers Unsupervised Learning and Reinforcement Learning, since these are the relevant paradigms for self-learning systems.

Objectives:

Neural networks are a fascinating interdisciplinary field where physicists, biologists, and computer scientists work together in order to better understand the information processing in biology. In this course paradigms of unsupervised learning and reinforcement learning are discussed from a biological point of view and analyzed mathematically. At the end of the semester, the students will be able to understand, implement use, and refine major algorithms of unsupervised and reinforcement learning

Contents:

• Week 1 - Unsupervised learning as opposed to supervised or reinforcement learning; Neurons and Synapses, Biology of unsupervised learning, Hebb rule and Long-Term Potentiation
• Week 2 - Hebb rule in a linear neuron model, Principal Component Analysis (PCA), and Oja's rule
• Week 3 - Introduction to Independent Component Analysis (ICA)
• Week 4 - Plasticity in a nonlinear neuron model and Independent Component Analysis (ICA)
• Week 5 - Competitive Learning and Clustering
• Week 6 - Introduction to Kohonen networks
• Week 7 - Analysis of Hebb rule and application to developmental learning

• Week 8 - The paradigm of reward-based learning in biology and theoretical formalisation; Reinforcement learning in discrete spaces
• Week 9 - Eligibility traces and reinforcement learning in continuous spaces
• Week 10 - Application of Reinforcement Learning: Navigation of rodents and Morris water maze
• Week 11 - Policy gradint rule for reinforcement learning
• Week 12 - Reward-modulated plasticity: Spike-Timing Dependent Plasticity, Neuromodulators and Learning and current research.
• Week 13 - Unsupervised learning in the ventral stream, object recognition, invariances, Slow Feature Analysis.
• Week 14 - Introduction ot Neuroeconomics

• Spiking Neuron Models by W. Gerstner and W. Kistler (Cambridge Univ. Press).
• Theoretical Neuroscience by P. Dayan and L.F. Abbott (MIT Press).

Exercises.
Exercise sessions are normally separate from the lectures, but may occasionally be integrated in the lectures.