Cryogenic Electronic Tomography (Cryo-ET) has become a major imaging technique that allows to reconstruct biological structure of high importance for biologists. This revolutionary  technique is a direct extension of Cryogenic Electronic Microscopy (Cryo-EM) that has been awarded the Nobel prize of Chemistry in 2017.
This imaging technique allows to recover large cell structures with resolution of only few nanometer. However, this imaging technique face numerous computational challenges, see Figure 1.
First, due to physical constraint, the electron beam can only probe a fraction of the entire volume. The inverse problem, recovering the 3-dimensional volume based on the 2-dimensional projections, is ill-posed. This missing information leads to strong artifacts of the 3-dimensional reconstruction. This problem is very similar to the problem of limited view computed tomography.
Second, the 2-dimensional observations are affected by random transformations. These deformation can be global such as shifts or rotations, but also local due to beam induced deformation. Aligning all the projections together is essential to get the best possible resolution available with the microscope and to avoid artifacts in the reconstruction, see Figure 2.
And finally, the observed images suffer from a large amount of noise. Each projection typically has a signal to noise ration (SNR) up to -20dB.

 

 

Figure 1: Illustration of single-particle cryo-EM imaging. We observe projections of the volume density along random directions(left). We observe a large number of projections (up to 100,000) that are degraded by a lot a random noise (middle). Cryo-EM reconstruction algorithm aims at recovering a sharp estimation of the 3-dimensional molecular structure.

Figure 2: Left: Three consecutive projections. You can see that they are not aligned, which cause the reconstruction algorithm to fail if not taken into account. Right: The target molecule to recover.

 


In the past years, learning based methods have become a major tool to solve many signal processing problem. The field of cryo-EM and cryo-ET has been nearly spared by such methods. There are several reason for that, like the necessity to deal with a large amount of data and the absence of clean training dataset.
In this project, we will explore how to use recent tools from machine learning to help improving the reconstruction in cryogenic tomography.

 

 

Project description

Depending on the interest of the student, we will explore at least one of the following perspective.

Generative model for 3-dimensional tomography

The main reason why deep learning methods have not yet made a breakthrough in solving the cryo-ET problem is the lack of a clean dataset of 3-dimensional molecules. Another significant limitation is the large amount of memory needed to store and process this volumes.
In the meanwhile, generative models have been refined a lot and allows to construct informative prior that helps at solving difficult imaging problems.

In this project, we will explore how generative model can be used to infer meaningfully prior for 3-dimensional reconstruction in cryo-ET. This project requires strong coding and few mathematical skills.

Estimation of unknown pose parameters

In cryo-ET, each projection can be seen as a deformed version of the clean one. Accounting for this deformation, also refer as a registration step, allows to reconstruct 3-dimensional volumes of much better quality. However, this step requires a lot of resources and very often relies on empirical methods.

In this project, we propose to build on a recent work conducted in the group [1] and estimate a more accurate family of deformation, accounting for global (shift, rotation) and local deformations.
This project requires strong coding and few mathematical skills.

Continuous representation for tomography

Implicit neural representation aims at representing a function $x\mapsto f(x)$ by a neural network taking the position $x$ as input.
It has been successfully applied to solve several challenging imaging problems such as in cryo-ET [2]. In this previous work, the implicit representation was used to reproduce the measurement into the image space. However, due to the particular structure of the cryo-ET problem and what is called the Fourier-slice theorem, it is also natural to fit the measurements into the Fourier space.

In this project, we will explore the benefit of implicit measurement in more detail using the particular structure of the cryo-ET problem.
It can also be combined with another recent work conducted in our group about a continuous generative model [3]. This project requires strong coding and few mathematical skills.

Organisation

Any of the above project can be adapted into either a Bachelor or a Master thesis.
Depending on the difficulty of the project, we will start with simpler applications such as (limited-view) computed tomography.

Depending on the chosen project, you will be supervised by at least two different person. Please reach out with one of the following for a first inquiry:

  • Valentin Debarnot, valentin.debarnot[@]unibas.ch
  • Ivan Dokmanić, ivan.dokmanic[@]unibas.ch

Bibliography

[1] Sidharth Gupta, Konik Kothari, Valentin Debarnot, and Ivan Dokmani ́c. Differentiable uncali-
brated imaging. arXiv preprint arXiv:2211.10525, 2022.

[2] Valentin Debarnot, Sidharth Gupta, Konik Kothari, and Ivan Dokmanic. Joint cryo-et alignment
and reconstruction with neural deformation fields. arXiv preprint arXiv:2211.14534, 2022.

[3] Amir Khorashadizadeh, Anadi Chaman, Valentin Debarnot, and Ivan Dokmani ́c. Funknn: Neu-
ral interpolation for functional generation. 2022.