The emerging field of geometric deep learning has put a spotlight on the role of symmetry and low-dimensional latent structures in high-dimensional data analysis. In this course we start by discussing the curse of dimensionality which precludes high-dimensional learning without latent structure. We then exploit symmetry: we review the basics of group representation theory and design convolutional neural networks from first principles, starting with convnets on grids and progressing over homogeneous spaces (in particular the sphere) to graphs. Finally, we connect graphs and manifolds via the Laplace–Beltrami operator and discuss the basics of spectral graph theory and graph-based clustering and manifold learning algorithms.

Timetable

Date Topic Resources Assignment
Monday 21.02.2022

No class

 

 
Thursday 24.02.2022 Introduction

GGGGG Section 2.1 - 2.2
(Donoho, 2000)

 
Monday 28.02.2022 Basics of statistical learning Lecture 2 of the GDL course   
Thursday 03.03.2022 Basics of statistical learning

Lecture 2 of the GDL course;

Chapter 2 of Shalev-Shwartz and Shai Ben-David (2014)

 
Monday 07.03.2022 Fasnachtsferien    
Thursday 10.03.2022 Fasnachtsferien    
Monday 14.03.2022 ERM on finite hypothesis classes  2.3.1 of Shalev-Shwartz and Shai Ben-David (2014)  
Thursday 17.03.2022 ERM recap; basics of groups The Symmetry roup of Isosceles triangleDihedral groups  
Monday 21.03.2022 Invariances and Equivariance on sets and graphs

Section 4.1 of GGGGG

 
Thursday 24.03.2022 Group representations; group invariance & equivariance Section 1-3 of our lecture notes  
Monday 28.03.2022 Recap of group representations, invariance, and equivariance Section 1-3 of our lecture notes  
Thursday 31.03.2022 Building group invariant & equivariant functions Section 4 of our lecture notes Problem Set 1
Jupiter-notebook--for-problem-C4 
 Monday 04.04.2022 Colab and ConvNet tutorials Colab tutorialConvNet tutorial  
Thursday 07.04.2022 Fourier transform and group representations Notes by Terence Tao  
Monday 11.04.2022 Guest lecture: Truly shift-invariant CNNs Truly shift-invariant CNNs  
Thursday 14.04.2022

Ostern

   
Monday 18.04.2022 Ostern    
Thursday 21.04.2022 From Fourier transform to Wavelet transform Section 4 and 5 of our lecture notes  
Monday 25.04.2022 Section A and B of the first exercise Solutions to problems were sent via email  
Thursday 28.04.2022 Wavelet transform; Spherical CNNs    
Monday 02.05.2022 Demo of Fourier and Wavelet transforms;
Homework solutions

Colab demo of the Fourier and Wavelet transforms

Solutions to problems were sent via email.

 
Thursday 05.05.2022 No class Reading material  
Monday 09.05.2022 Spherical CNNs; exercises    
Thursday 12.05.2022 From Spherical CNN to graph neural nets    
Monday 16.05.2022 Homework exercises

Implementing a CNN classifier using pytorch

List of project papers

Team formation

 
Thursday 19.05.2022      
Monday 23.05.2022      
Thursday 26.05.2022     Exercise sheet 3 (optional)
Monday 30.05.2022      
Thursday 02.06.2022      

 

 

 

Resources

 

Contact

Lecturer

Prof. Dr. Ivan Dokmanić: ivan.dokmanic[at]unibas.ch

 

Teaching assistant

Tianlin Liu: t.liu[at]unibas.ch