A hands-on introduction to modern pattern recognition. We begin in medias res by training two intuitive classifiers (a perceptron and a kNN classifier) to recognize digits. To determine which one is better we must state what "better" means, so we introduce "generalization." To talk about generalization we need to build the probabilistic framework in which modern pattern recognition resides. From here the plan is clear: we imagine that probabilistic descriptions of data (that is, the knowledge) is given to us and derive optimal classifiers. We then move to supervised learning from training samples, linear classification and regression, feature design (especially the multiscale or spectral perspective which has seen great success for images, audio, text, ..., and which has beautiful connections to physics). We apply our probabilistic models to problems in image restoration—denoising, demosaicking, deblurring, as well as—time permitting—segmentation. We then eschew feature design in favor of deep neural networks, where we will focus on convolutional architectures and transformers whose attention mechanisms learn representations end-to-end across modalities. Finally, we pivot back to the generative question: what good are probabilistic descriptions of the world if we cannot produce samples from them? This leads us to diffusion models—turning noise into data—and to the unifying links between estimation, optimization, and dynamics that make them work.

For discussions about the lectures and excercises you can use the Piazza forum. Please do not send e-mail unless you're asking a really personal question. 

Schedule

W Topic Additional material Problem sets
1

T: Introduction: the ingredients (slides, video 1, video 2) 
F: empirical risk minimization, k-nearest neighbors algorithm, curse of dimensionality (video 1, video 2)

Recitation: A refresher on linear algebra and probability (ipynb)

Week 1 notes

Introductory problem set + model solutions
2

T: no lecture
F: knn, linear functions (video 1, video 2)

Recitation: Introductory problem set

Week 2 notes

Problem set 1 + model solutions
3

T: linear regression, logistic regression, perceptron (video 1, video 2)
F: multivariate regression, gradient descent, convolution (video 1, video 2)

Recitation: open Q & A

Week 3 notes (part 1)

 

 
4

T: Deep learning bootcamp part 1 
F: Deep learning bootcamp part 2 

Lecture notes: 

Recitation: 

 

Problem set 2
5

T: 
F: 

Lecture notes: 

Recitation: 

 

 
6

T: 
F: 

Lecture notes:

Recitation: 

 

 
7

T: 
F: 

Lecture notes: 

Recitation: 

 

 
8

T: 
F:  

Lecture notes: 

Recitation: 

 

 
9

T: 
F: 

Lecture notes: 

Recitation: 

 

 

 
10

T: 
F: 

Lecture notes: 

Recitation: 

 

 
11

T: 
F:

Lecture notes: 

Recitation: 

 

 
12

T: 
F: 

Lecture notes: 

Recitation: 

 

 
13

T: 
F: 

Lecture notes: 

Recitation: 

 

 
14

T: 
F: 

Lecture notes: 

Recitation: 

 

 

Prerequisites

We will assume familiarity with basic probability and linear algebra. We will review the most important concepts along the way but these reviews cannot replace entire courses. All programming examples and assignments will be in Python.

Resources

The course materials will be posted to the ADAM platform. While we will provide the required notes and recordings, we will also recommend reading from textbooks and other sources that are freely available online. 

Main Texbook:

[FOC] Foundations of Computer Vision (2024) by Torralba, Isola and Freeman: A great textbook that this lecture will be mainly based on.

Additional References:

[MML] Machine Learning: A Probabilistic Perspective (2012) by Kevin Murphy; see also newer books by the same author, available online.

[ESL] Elements of Statistical Learning by Hastie, Tibshirani, and Friedman: A legendary ML textbook. See also a shorter introductory text by the same authors, Introduction to Statistical Learning.

[PPA] Patterns, Predictions, and Actions: A graduate textbook by Moritz Hardt and Ben Recht, somewhat advanced for this course but great reading nonetheless if you like that sort of stuff and you're not seeing it for the first time.

[FPS] Foundations of Signal Processing: A signal processing textbook by Vetterli, Kovaˇcevi´c and Goyal.

[CVAA] Computer Vision: Algorithms and Applications,  2nd Edition, by Richard Szeleski.

[ISL] An Introduction to Statistical Learning with Applications in Python by James, Witten, Hastie, Tibshirani and Taylor.

Math Resources:

The course uses a lot of probability and linear algebra; hence to be comfortable with the basics, we provide links to supplementary materials.  However, for a much more comprehensive set of resources, you can find them in  Jonathan Shewchuk's 'Introduction to Machine Learning' course. For your convinience, we have included some of those links below:

Contact

Lecturer

Prof. Dr. Ivan Dokmanić: ivan.dokmanic[at]unibas.ch

 

Teaching assistants

Felicitas Haag: felicitas.haag[at]unibas.ch

Alexandra Spitzer: alexandra.spitzer[at]unibas.ch

Vinith Kishore: vinith.kishore[at]unibas.ch

Cheng Shi: cheng.shi[at]unibas.ch

Class time and location

Lectures will be given in Kollegienhaus, Hörsaal 118

Tuesday lecture takes place from 08.15 am to 10.00 am. Friday lecture takes place from 10.15 am to 12.00 pm.

Exercise sessions will take place on Monday, 2.15 pm - 4.00 pm in Biozentrum, Seminarraum 02.073 and Wednesday, 4.15 pm - 6.00pm in Spiegelgasse 5, Seminarraum 05.002.

 

 

Grade

First midterm exam 25%
Second midterm exam 25%
Final exam 50%

Exam dates

There will be two midterm exams, and one final exam.

The first midterm exam will take place on 21.10.2025 during regular lecture hours.
The second midterm exam will take place on 02.12.2025 during regular lecture hours.

The final exam will take place on 20.01.26, 14-17 h.