Convolutional sparse coding (CSC) based methods have been shown to provide excellent performance on image reconstruction tasks like super-resolution, MRI/CT reconstruction, de-raining, etc [1,2,3]. These approaches generate sparse representatios for images using convolutional dictionaries whose parameters can be trained in a task-driven manner over a dataset.
While the performance of traditional CSC based methods was not as effective as deep neural networks, recently proposed dictionary learning approach called ISTA U-Net  showed comparable performance to the U-Net on various image reconstruction tasks. In addition, unlike U-Net, the ISTA-UNet offers an additional advantage of being interpretible from a signal processing perspective.
Figure 1: Schematic illustration of the classic U-Net (left panel) and the dictionary model of the ISTA U-Net.
The image reconstruction tasks over which U-Net and ISTA UNet are known to be comparable are all convolutional in nature, i.e. the input and output images can be related via a convolution. However, U-Net has been shown to perform well even on highly non-convolutional problems, for instance, XOR decryption .
In this project, we have the following goals.
- We would like to evaluate the performance of dictionary learning methods (especially ISTA U-Net) on such non-convolutional tasks and compare them against U-Net.
- We would investigate the factors that enable the performance of U-Net on such tasks and experimentally assess the limits of the network's performance. Similarly, we would attempt to find the reasons for any differences between U-Net and ISTA U-Net.
This project would have a significant coding component. The student should be proficient in training deep learning models in pytorch/tensorflow. Additional background in signal processing would be helpful, though not necessarily required.
Interested students are requested to contact Anadi Chaman.
 S. Gu, W. Zuo, Q. Xie, D. Meng, X. Feng and L. Zhang, "Convolutional Sparse Coding for Image Super-Resolution," 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1823-1831, doi: 10.1109/ICCV.2015.212.
 M. Li et al., "Video Rain Streak Removal by Multiscale Convolutional Sparse Coding," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 6644-6653, doi: 10.1109/CVPR.2018.00695.
 T. Liu, A. Chaman, D. Belius and I. Dokmanić, "Learning Multiscale Convolutional Dictionaries for Image Reconstruction," in IEEE Transactions on Computational Imaging, vol. 8, pp. 425-437, 2022, doi: 10.1109/TCI.2022.3175309.
 Hauptmann, Andreas, and Jonas Adler. "On the unreasonable effectiveness of CNNs." arXiv preprint arXiv:2007.14745(2020).