Instructors: Prof. Dr. Nassir Navab, Dr. Shahrooz Faghihroohi, Azade Farshad, Yousef Yeganeh
Time: TBA
Registration
- Registration must be done through TUM Matching Platform (please pay attention to the Deadlines)
- In order to increase your priority, please also apply via our own Registration system.
- The maximum number of participants: 20.
Announcements
- The presentation and blogpost guidelines are available here: TBA
- The preliminary meeting slides can be found here: DLMA-PreliminaryMeeting-SS24.pdf
- The preliminary meeting is scheduled for Feb 1st, 13:30 to 14:00 with the following Zoom link:
https://tum-conf.zoom-x.de/j/62875182420?pwd=SWlFM0Jya0dQVFNLeUVrUHg5cWhDUT09
Introduction
- Deep Learning is growing tremendously in Computer Vision and Medical Imaging as well. Highly impacted journals in the medical imaging community, i.e. IEEE Transaction on Medical Imaging, recently published their special edition on Deep Learning [1]. The Seminar will propose a list of recent scientific articles related to the main current research topics in deep learning for Medical Applications, together with some interesting papers from other communities (CVPR, NeurIPS, ICCV, ICLR, ICML, ...).
Course Structure
In this Master Seminar (Hauptseminar), students select one scientific topic from the list provided by course organizers. The students should read the proposed sample papers by the tutors, find the topic-related articles, summarize and compare them in their presentation and blogpost:
- Presentation: The selected paper is presented to the other participants (Maximum 25 minutes presentation, 10 minutes questions). You can use the CAMP templates for PowerPoint TUM-Template.pptx.
- Blog Post: A blog post of 3000-3500 words excluding references, should be submitted before the deadline. The blog post must include all references used and must be written completely in your own words. Copy and paste will not be tolerated.
- Attendance: Participants have to participate actively in all seminar sessions. Each presentation is followed by a discussion, and everyone is encouraged to actively participate.
Submission Deadline: You have to submit the blog post by your presentation date and can modify it until the last session of the course.
Schedule (TBA)
| Date | Session: Topics | Students |
|---|---|---|
| 13.06.2024 | Multimodal Generative Models Causal Generative Models Handling Motion in Medical Imaging with Spatio-Temporal Generative Models | Linus Salzmann Simoleit Cameron Julien Schulz |
| 20.06.2024 | Wavelet and Diffusion Models Medical Image Reconstruction Using Diffusion Models Diffusion-based 3D Shape completion | Leonhard Zirus Janina Schellenberg Johannes Thyroff |
| 27.06.2024 | Video Synthesis Using Diffusion Model Graph Diffusion Models Fast diffusion models | Yugay Vasiliy UNG Jacques Zhang Shihong |
| 04.07.2024 | Deformable Image Registration with Implicit Neural Representations Neural Implicit Representations for Medical Shapes 3d reconstruction in the context of medical applications | Zhang Xingyu Laura Leschke Arpi Arustamyan |
| 11.07.2024 | Can a Neural Network learn Physiology? Fine-tuning Large Language Models using Reinforcement Learning | Donnate Hooft Tomislav Pavković |
List of Topics and Material
The proposed papers for each topic in this course are usually selected from the following venues/publications:
CVPR: Conference on Computer Vision and Pattern Recognition
ICLR: International Conference on Learning Representations
NeurIPS: Neural Information Processing Systems
TPAMI: IEEE Transactions on Pattern Analysis and Machine Intelligence
TMI: IEEE Transaction on Medical Imaging
JBHI: IEEE Journal of Biomedical and Health Informatics
MedIA: Medical Image Analysis (Elsevier)
MICCAI: Medical Image Computing and Computer-Assisted Intervention.
BMVC: British Machine Vision Conference
MIDL: Medical Imaging with Deep Learning
List of papers (TBA)
| No | Topic | Sample Papers | Journal/ Conference | Tutor | Student | Link |
|---|---|---|---|---|---|---|
| 1 | Multimodal Generative Models | Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models | CVPR 2024 | Linus Salzmann | https://arxiv.org/pdf/2404.03913 | |
| Instruct-Imagen: Image Generation with Multi-modal Instruction | CVPR 2024 | https://arxiv.org/abs/2401.01952 | ||||
| MedM2G: Unifying Medical Multi-Modal Generation via Cross-Guided Diffusion with Visual Invariant | CVPR 2024 | https://arxiv.org/abs/2403.04290 | ||||
| 2 | Causal Generative Models | Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting | ICML 2023 | Simoleit Cameron | https://openreview.net/pdf?id=Kw90j2pNSt | |
| Causal-CoG: A Causal-Effect Look at Context Generation for Boosting Multi-modal Language Models | CVPR 2024 | https://arxiv.org/abs/2312.06685 | ||||
| What If the TV Was Off? Examining Counterfactual Reasoning Abilities of Multi-modal Language Models | ICCV 2023 | https://arxiv.org/abs/2310.06627 | ||||
| 3 | Graph Diffusion Models | Data-Centric Learning from Unlabeled Graphs with Diffusion Model | NeurIPS 2023 | UNG Jacques | ||
| Autoregressive Diffusion Model for Graph Generation | ICML 2023 | https://proceedings.mlr.press/v202/kong23b/kong23b.pdf | ||||
| DiGress: Discrete Denoising diffusion for graph generation | ICLR 2023 | https://arxiv.org/abs/2209.14734 | ||||
| 4 | Medical Image Reconstruction Using Diffusion Models | DiffGAN: An adversarial diffusion model with local transformer for MRI reconstruction | Magnetic Resonance Imaging 2024 | Shahrooz Faghihroohi | Janina Schellenberg | https://www.sciencedirect.com/science/article/pii/S0730725X24000730 |
| DOLCE: A model-based probabilistic diffusion framework for limited-angle ct reconstruction | ICCV 2023 | |||||
| Adaptive diffusion priors for accelerated MRI reconstruction | MedIA 2023 | https://www.sciencedirect.com/science/article/pii/S1361841523001329 | ||||
| 5 | Video Synthesis Using Diffusion Model | Align your latents: High-resolution video synthesis with latent diffusion models | ICCV 2023 | Shahrooz Faghihroohi | Yugay Vasiliy | |
| Structure and content-guided video synthesis with diffusion models | ICCV 2023 | |||||
| Lumiere: A space-time diffusion model for video generation | Arxiv 2024 | https://arxiv.org/html/2401.12945v2 | ||||
| 6 | Handling Motion in Medical Imaging with Spatio-Temporal Generative Models | CHeart: A Conditional Spatio-Temporal Generative Model for Cardiac Anatomy | TMI 2023 | Julien Schulz | https://ieeexplore.ieee.org/document/10315018 | |
| Learning a Generative Motion Model From Image Sequences Based on a Latent Motion Matrix | TMI 2021 | https://ieeexplore.ieee.org/document/9344838 | ||||
| SADM: Sequence-Aware Diffusion Model for Longitudinal Medical Image Generation | IPMI 2023 | https://arxiv.org/pdf/2212.08228 | ||||
| 7 | Deformable Image Registration with Implicit Neural Representations | Deformable Image Registration with Geometry-informed Implicit Neural Representations | MIDL 2024 | Zhang Xingyu | https://proceedings.mlr.press/v227/harten24a/harten24a.pdf | |
| SINR: Spline-enhanced implicit neural representation for multi-modal registration | MIDL 2024 | https://openreview.net/pdf?id=V5XDYSRcQP | ||||
| Robust Deformable Image Registration Using Cycle-Consistent Implicit Representations | TMI 2023 | |||||
| 8 | Wavelet and Diffusion Models | Wavelet-Improved Score-Based Generative Model for Medical Imaging | TMI 2024 | Mohammad Farid Azampour | Leonhard Zirus | |
| Wavelet Score-Based Generative Modeling | NeurIPS 2022 | |||||
| Neural Wavelet-domain Diffusion for 3D Shape Generation | Siggraph 2022 | https://arxiv.org/pdf/2209.08725.pdf | ||||
| 9 | Neural Implicit Representations for Medical Shapes | 4D Myocardium Reconstruction with Decoupled Motion and Shape Model | ICCV 2023 | Laura Leschke | ||
| MedShapeNet - A Large-Scale Dataset of 3DMedical Shapes for Computer Vision | Arxiv 2023 + MICCAI Workshop | https://arxiv.org/pdf/2308.16139 | ||||
| ImplicitAtlas: Learning Deformable Shape Templates in Medical Imaging | ICPR 2022 | |||||
| 10 | Diffusion-based 3D Shape completion | SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation | CVPR 2023 | Miruna-Alexandra Gafencu | Johannes Thyroff | |
| Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions | ICCV 2023 | |||||
| 3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models | TOG 2023 | https://dl.acm.org/doi/pdf/10.1145/3592442 | ||||
| 11 | Fine-tuning Large Language Models using Reinforcement Learning | Training language models to follow instructions with human feedback | NeurIPS 2022 | David Bani-Harouni | Tomislav Pavković | https://arxiv.org/abs/2203.02155 |
| Quark: Controllable Text Generation with Reinforced [Un]learning | NeurIPS 2022 | https://arxiv.org/abs/2205.13636 | ||||
| Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering | EMNLP 2022 | https://arxiv.org/abs/2210.03078 | ||||
| 12 | Can a Neural Network learn Physiology? | The New Field of Network Physiology: Building the Human Physiolome | Frontiers in Network Physiology 2021 | Francesca De Benetti | Donnate Hooft | https://www.frontiersin.org/articles/10.3389/fnetp.2021.711778/full |
| Dynamic networks of cortico-muscular interactions in sleep and neurodegenerative disorders | Frontiers in Network Physiology 2023 | https://www.frontiersin.org/articles/10.3389/fnetp.2023.1168677/full | ||||
| Dynamic networks of physiologic interactions of brain waves and rhythms in muscle activity | Human Movement Science 2022 | https://www.sciencedirect.com/science/article/pii/S0167945722000513 | ||||
| 13 | 3d reconstruction in context of medical applications | Benchmarking Encoder-Decoder Architectures for Biplanar X-ray to 3D Shape Reconstruction | NeurIPS2024 | Agnieszka Tomczak | Arpi Arustamyan | |
| 3D reconstruction of proximal femoral fracture from biplanar radiographs with fractural representative learning | Scientific Reports 2023 | https://www.nature.com/articles/s41598-023-27607-2 | ||||
| A Deep-Learning Approach For Direct Whole-Heart Mesh Reconstruction | Medical Image Analysis 2021 | https://arxiv.org/abs/2102.07899 | ||||
| 14 | Fast diffusion models | UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models | Neurips 2024 | Mohammad Farid Azampour | Zhang Shihong | |
| Fast Sampling of Diffusion Models via Operator Learning | ICML 2023 | https://proceedings.mlr.press/v202/zheng23d/zheng23d.pdf | ||||
| One-Step Diffusion Distillation via Deep Equilibrium Models | Neurips 2024 |
Literature and Helpful Links
A lot of scientific publications can be found online.
The following list may help you to find some further information on your particular topic:
- Microsoft Academic Search
- Google Scholar
- CiteSeer
- CiteULike
- Collection of Computer Science Bibliographies
Some publishers:
- ScienceDirect (Elsevier Journals)
- IEEE Journals
- ACM Digital Library
Libraries (online and offline):
- http://rzblx1.uni-regensburg.de/ezeit/ (Elektronische Zeitschriften Bibliothek)
- Verbundkatalog des Bibliotheksverbundes Bayern (BVB)
- Computer ORG
- http://www.ub.tum.de/ (TUM Library)
- To get access onto the electronic library, see http://www.ub.tum.de/medien/ejournals/readme.html
- "proxy.biblio.tu-muenchen.de" mit Port 8080 (nur fuer http). Damit klappen zumindest portal.acm.org und computer.org meistens
- Various proceedings of conferences in our AR-Lab, 03.13.036 (These proceedings are not for lending!)
Some further hints for working with references:
- JabRef is a Java program for comfortable working with Bibtex literature databases. Handy feature: if you know the PubMed ID for an article, JabRef can import data from there (via "Web Search/Medline").
- Mendeley is a cross-platform program for organising your references.
If you find useful resources that are not already listed here, please tell us, so we can add them for others. Thanks.