Laboratory of Computational Neuroscience
LCN - Brain Mind - I&C - Life Sciences - EPFL

Video Lectures of Wulfram Gerstner:

Artificial Neural Networks and Deep Learning

• Introduction Supervised learning, classification, simple perceptron (62 min)
  Part 1 - Classification as a geometric problem (9 min)
  Part 2 - Supervised learning (5 min)
  Part 3 - Simple Perceptron (14 min)
  Part 4 - Perceptron algorithm (12 min)
  Part 5 - Stochastic gradient descent algorithm (22 min)

  slides for Introduction - Lecture 0: Classification, Supervised Learning, Simple Perceptron

• Deep Learning Lecture 1. Multilayer Networks, BackProp and Regularization (80 min)
  (These videos are also on this earlier channel )
  Part 1 - Gradient Descent Methods (13 min)
  Part 2 - XOR problem (5 min) also here on youtube
  Part 3 - Multilayer Networks: First Steps (15 min) also here on youtube
  Part 4 - BackPropagation Algorithm (10 min) also here on youtube
  Part 5 - The Problem of Overfitting (12 min)
  Part 6 and 7 - Simple Regularization Methods, Training Base and Validation Base (13 min)
  Part 8 - Careful Crossvalidation (4 min)
  Part 9 - Early Stopping (8 min)
  

  slides for DeepLearning - Lecture 1: MultiLayer Networks and BackProp

• Deep Learning Lecture 2. Tricks of the Trade in Deep Leanring (85 min)
  Attention: If your computer is on mode 'Stereo' you will hear (in some videos) the sound only in the left ear when using headphones. If you want the voice in your headphones bilaterally, please switch your audio-settings on your computer to 'Mono'. (Under windows, search for 'mono')
  Part 1 - Questions and Aims (6 min)
  Part 2 - Bagging (7 min)
  Part 2* - Theory of Bagging (12 min)
  Part 3 - Dropout (8 min)
  Part 4 - Data augmentation (6 min)
  Part 5 - Weight initialization (19 min)
  Part 6 - Vanishing Gradient Problem (9 min)
  Part 7 - Weight update: mean input and bias problem (11 min)
  Part 8 - Batch normalization (7 min)
  

  slides for DeepLearning - Lecture 2: Tricks of the Trade

• Deep Learning Lecture 3. Loss Landscape and optimization methods for deep learning ( 101 min)

  Part 1 - Questions and aims of this lecture (6 min)
  Part 2 - Loss Landscape: minima and saddle points (14 min)
  Part 3 - Why are there so many saddle points? (28 min)
  Part 4 - Momentum (15 min)
  Part 5 - RMSprop and ADAM (21 min)
  Part 6 - No Free Lunch theorem (8 min)
  Part 7 - Deep networks versus shallow networks (9 min)
  

  slides for DeepLearning - Lecture 3: Loss Landscape and Optimization Methods for Deep Learning

• Deep Learning Lecture 4. Statistical Classification by Neural Networks (90 min)

  Part 1 - The statistical view: generative models (6 min)
  Part 2 - The likelihood of data under a model (12 min)
  Part 3A - Statistical interpretation of artificial neural networks: the cross-entropy loss function (19 min)
  Part 3B - Statistical interpretation of artificial neural networks: can we interpret the output as a probability? (18 min)
  Part 4 - Sigmoidal units as natural output functions (9 min)
  Part 5 - Multi-class problems (19 min)
  Part 6 - Statistical approach: Summary and Quiz (7 min)
  

  slides for DeepLearning - Lecture 4: Stastical Classification by Deep Networks

• Deep Learning Lecture 5. Convolutional Neural Networks (110 min)

  Part 1 - Inductive bias in machine learning (11 min)
  Part 2 - Convolution filters as inductive bias for images (16 min)
  Part 3 - MaxPooling as inductive bias for images (20 min)
  Part 3* - Quiz (6 min)
  Part 4 - The gradient of a convolutional layer: (16 min)
  Part 5 - Automatic differentiation: BackProp revisted (9 min)
  Part 6 - Reducing the number of parameters: outer-product (8 min)
  Part 7 - Modern ConvNets and image recognition tasks (17 min)
  Part 8 - Applications beyond object recognition (9 min)
  

  slides for DeepLearning - Lecture 5: Convolutional Networks

• Deep Learning, Ian Goodfellow et al. (MIT Press)
• Neural Networks for Pattern Classification , Chris Bishop (Oxford Univ. Press)