Access Type
Open Access Embargo
Date of Award
January 2022
Degree Type
Dissertation
Degree Name
Ph.D.
Department
Computer Science
First Advisor
Ming Dong
Abstract
Deep learning methods have achieved great success in different areas of Computer Vision including image-to-image translation. Image-to-image translation is to learn a mapping between images from a source domain and images from a target domain and has many applications including image colorization, generating semantic labels from images, image super resolution, medical image synthesis, and domain adaptation.Many image-to-image translation approaches require supervised learning settings in which pairs of corresponding source and target images are available. However, acquiring paired training data is expensive or sometimes impossible for diverse applications. Therefore, there are approaches in unsupervised settings in which, source and target image sets are completely independent with no paired examples between the two domains. Furthermore, there are many applications of image-to-image translation contains paired training data with inconsistent regions. Containing both consistent and inconsistent regions in the training data makes these applications challenging problems in image-to-image translation task. There are many examples of inconsistent data in medical imaging tasks due to differences in respiratory or physiological states. In this proposal, we focus on image-to-image translation task with deep neural networks for different learning settings including supervised, unsupervised, combination of supervised and unsupervised settings on inconsistent datasets, and self-supervised setting. First, we present a novel supervised image-to-image translation model to generate synthetic CT images from MRI data based on generative adversarial networks. The proposed approach shows that incorporating adversarial learning generates more realistic synCTs with higher spatial resolution and lower MAE than CNN with no discriminator block. Our experimental results show that the proposed model can efficiently and accurately generate synCT images from the MRI input while outperforming state-of-the-art models, thus offering strong potential for supporting MR-only radiation therapy workflows. Second, we introduce the attention directly to the generative adversarial network (GAN) architecture and propose a novel spatial attention GAN model (SPA-GAN) for image-to-image translation tasks. The proposed SPA-GAN model computes the attention in its discriminator and use it to help the generator focus more on the most discriminative regions between the source and target domains, leading to more realistic output images. Through extensive experiments in both supervised and unsupervised settings, we demonstrate that our framework outperforms the current state-of-the-arts. Third, we propose a novel deep learning framework for translating MRI to CT domain on challenging inconsistent dataset. We handle the training data inconsistency by introducing two-stream architecture that can improve organ structures in the generated synthetic CTs. We also propose to learn the style of inconsistent regions in CT domain by using style transfer modules. Our experimental results show that our proposed framework can handle the region inconsistency in the training data and generate accurate synthetic CTs. Forth, we proposed a novel self-supervised image-to-image translation framework using contrastive learning for datasets with limited paired data. To overcome the problem of limited paired data in many applications including clinical applications, the need for large number of paired training samples is removed by incorporating contrastive learning in image-to-image translation framework. The proposed CL-GAN achieved superior performance of synthesizing PET images from MRI, both qualitative and quantitative, over current state-of-the-arts. The effectiveness of CL-GAN is more evident when only a small number of paired training data is available.
Recommended Citation
Emami, Hajar, "Image-To-Image Translation With Deep Neural Networks" (2022). Wayne State University Dissertations. 3733.
https://digitalcommons.wayne.edu/oa_dissertations/3733