Access Type
Open Access Embargo
Date of Award
January 2023
Degree Type
Dissertation
Degree Name
Ph.D.
Department
Computer Science
First Advisor
Weisong Shi
Abstract
Deep neural networks (DNNs) are widely used in autonomous driving due to their high accuracy for perception, decision, and control. Predictability of the perception module is essential for the AV's safety. Predictability generally consists of two aspects: temporal and functional. Temporal aspects mean the task should be finished before the deadline. Functional aspects mean the task should make correct decisions. However, non-negligible time and performance variations are observed in DNN inference. Current DNN inference studies either ignore the variation issue or rely on the scheduler or the algorithm itself to handle it. None of the current work explains the roots causing DNN inference variations.
In this dissertation, we explore the predictability of DNN inference in AV with an experimental approach. For DNN inference time variations, we did fine-grained time profiling of typical DNN models. We derived six insights into the relationship between DNN inference time variations and variability of data, I/O, model, runtime, hardware, and end-to-end perception system. Based on the observations on the DNN model's structure and multi-tenant DNN coordination, we propose Prophet, which solves the time variations in a predictable AV perception pipeline in two steps: predict the time variations based on the intermediates results like proposals and raw points, and coordinate the multi-tenant DNNs to ensure the inference progress is close to each other. From the evaluation results on the KITTI dataset, the time prediction of a single model achieves higher than 91% for Faster R-CNN, LaneNet, and PINet. Besides, the perception fusion delay is bounded to 150ms, and the fusion drop ratio is reduced from 5.4% to 0.085%.
For DNN inference performance variations, we got two insights on addressing the predictability issue for the perception pipeline from empirical studies on a real AV pipeline. Firstly, there is a high temporal locality in image streaming detection. Secondly, out-of-date frames in the message queue contribute to the huge fusion delay. Therefore, we propose DeepReferee, a general framework to guarantee performance predictability for multi-tenant DNN inference in AV's perception pipeline. DeepReferee comprises three modules: the keyframe selector, the detection predictor, and the frame dispatcher. From the evaluation with the BDD100K dataset, DeepReferee improves the number of fusion frames by 3x, reduces the fusion delay by 51%, and improves the detection completeness by 18% than the baseline.
Finally, to tackle a general approach to AV's predictability, we propose CPT, which is a configurable testbed for quantifying the predictability in AV's perception pipeline. CPT is composed of three modules: the application configurations, the system configurations, and the evaluator. CPT provides millions of configurations for the perception pipeline to study the predictability issue.
Recommended Citation
Liu, Liangkai, "Predictable Dnn Inference For Autonomous Driving" (2023). Wayne State University Dissertations. 3832.
https://digitalcommons.wayne.edu/oa_dissertations/3832
