The emerging field of geometric deep learning has put a spotlight on the role of symmetry and low-dimensional latent structures in high-dimensional data analysis. In this course we start by discussing the curse of dimensionality which precludes high-dimensional learning without latent structure. We then exploit symmetry: we review the basics of group representation theory and design convolutional neural networks from first principles, starting with convnets on grids and progressing over homogeneous spaces (in particular the sphere) to graphs. Finally, we connect graphs and manifolds via the Laplace–Beltrami operator and discuss the basics of spectral graph theory and graph-based clustering and manifold learning algorithms.
Timetable
Date | Topic | Resources | Assignment |
---|---|---|---|
Monday 21.02.2022 |
No class |
|
|
Thursday 24.02.2022 | Introduction |
GGGGG Section 2.1 - 2.2 |
|
Monday 28.02.2022 | Basics of statistical learning | Lecture 2 of the GDL course | |
Thursday 03.03.2022 | Basics of statistical learning |
Lecture 2 of the GDL course; Chapter 2 of Shalev-Shwartz and Shai Ben-David (2014) |
|
Monday 07.03.2022 | Fasnachtsferien | ||
Thursday 10.03.2022 | Fasnachtsferien | ||
Monday 14.03.2022 | ERM on finite hypothesis classes | 2.3.1 of Shalev-Shwartz and Shai Ben-David (2014) | |
Thursday 17.03.2022 | ERM recap; basics of groups | The Symmetry roup of Isosceles triangle; Dihedral groups | |
Monday 21.03.2022 | Invariances and Equivariance on sets and graphs |
Section 4.1 of GGGGG |
|
Thursday 24.03.2022 | Group representations; group invariance & equivariance | Section 1-3 of our lecture notes | |
Monday 28.03.2022 | Recap of group representations, invariance, and equivariance | Section 1-3 of our lecture notes | |
Thursday 31.03.2022 | Building group invariant & equivariant functions | Section 4 of our lecture notes | Problem Set 1 Jupiter-notebook--for-problem-C4 |
Monday 04.04.2022 | Colab and ConvNet tutorials | Colab tutorial; ConvNet tutorial | |
Thursday 07.04.2022 | Fourier transform and group representations | Notes by Terence Tao | |
Monday 11.04.2022 | Guest lecture: Truly shift-invariant CNNs | Truly shift-invariant CNNs | |
Thursday 14.04.2022 |
Ostern |
||
Monday 18.04.2022 | Ostern | ||
Thursday 21.04.2022 | From Fourier transform to Wavelet transform | Section 4 and 5 of our lecture notes | |
Monday 25.04.2022 | Section A and B of the first exercise | Solutions to problems were sent via email | |
Thursday 28.04.2022 | Wavelet transform; Spherical CNNs | ||
Monday 02.05.2022 | Demo of Fourier and Wavelet transforms; Homework solutions |
Colab demo of the Fourier and Wavelet transforms Solutions to problems were sent via email. |
|
Thursday 05.05.2022 | No class | Reading material | |
Monday 09.05.2022 | Spherical CNNs; exercises | ||
Thursday 12.05.2022 | From Spherical CNN to graph neural nets | ||
Monday 16.05.2022 | Homework exercises | ||
Thursday 19.05.2022 | |||
Monday 23.05.2022 | |||
Thursday 26.05.2022 | Exercise sheet 3 (optional) | ||
Monday 30.05.2022 | |||
Thursday 02.06.2022 |
Resources
- Lecture notes (work in progress)
- Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David
- Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges by Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković
- A course on Geometric Deep Learning
Contact
Lecturer
Prof. Dr. Ivan Dokmanić: ivan.dokmanic[at]unibas.ch
Teaching assistant
Tianlin Liu: t.liu[at]unibas.ch