Off-campus WSU users: To download campus access dissertations, please use the following link to log into our proxy server with your WSU access ID and password, then click the "Off-campus Download" button below.
Non-WSU users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
Access Type
WSU Access
Date of Award
January 2025
Degree Type
Dissertation
Degree Name
Ph.D.
Department
Computer Science
First Advisor
Ming Dong
Abstract
Though deep neural networks achieve great accuracy in visual recognition tasks, they contain millions of weights and thus require a large space to be stored. In this dissertation, we focus on compressing different types of deep neural networks in different situations. First, we present a novel deep compression method, Octave Deep Compression (ODC), to compress Octave Convolutional Networks with in-parallel pruning-quantization on different frequencies. Second, we propose a novel unstructured pruning pipeline, Attention-based Simultaneous sparse structure and Weight Learning (ASWL), where an efficient algorithm is proposed to calculate the pruning ratios layer-wisely from attentions, and both weights for the dense network and the sparse network are tracked so that the pruned structure is simultaneously learned from randomly initialized weights. Third, we focus on compressing and accelerating deep GCN models with residual connections using structured pruning by presenting AgileGCN. Specifically, in each residual structure of a deep GCN, channel sampling and padding are applied to the input and output channels of a convolutional layer, respectively, to significantly reduce its floating point operations (FLOPs) and number of parameters. Fourth, we propose a novel framework, Transferring Lottery Ticket (TLT), to adapt both masks and weights of a pre-trained and pruned network dynamically during the knowledge transfer to downstream tasks. Recent work has shown that pruned networks can also be used as pre-trained models in transfer learning. Finally, we propose MAGNET, a novel modality-agnostic network for 3D medical image segmentation, which is specifically designed to handle real medical situations where multiple modalities/sequences are available during model training, but fewer are available or used at the time of clinical practice.
Recommended Citation
He, Qisheng, "On Compressing Deep Neural Networks" (2025). Wayne State University Dissertations. 4237.
https://digitalcommons.wayne.edu/oa_dissertations/4237