Abstract

Extensive research has gone into optimizing convolutional neural network (CNN) architectures for tasks such as image classification and object detection, but research to date on the relationship between input image quality and CNN prediction performance has been relatively limited. Additionally, while CNN generalization against out-of-distribution image distortions persists as a significant challenge and a focus of substantial research, a range of studies have suggested that CNNs can be be made robust to low visual quality images when the distortions are predictable. In this research, we systematically study the relationships between image quality and CNN performance on image classification and detection tasks. We find that while generalization remains a significant challenge for CNNs faced with out-of-distribution image distortions, CNN performance against low visual quality images remains strong with appropriate training, indicating the potential to expand the design trade space for sensors providing data to computer vision systems. We find that the functional form of the GIQE can predict CNN performance as a function of image degradation, but we observe that the legacy form of the GIQE does a better job of modeling the impact of blur/relative edge response in some scenarios. Additionally, we evaluate other image quality models that lack the pedigree of the GIQE and find that they generally work as well or better than the functional form of the GIQE in modeling computer vision performance on distorted images. We observe that object detector performance is qualitatively very similar to image classifier performance in the presence of image distortion. Finally, we observe that computer vision performance tends to exhibit relatively smooth, monotonic variation with blur and noise, but we find that performance is relatively insensitive to resolution under a range of conditions.

Library of Congress Subject Headings

Imaging systems--Image quality; Deep learning (Machine learning); Computer vision; Neural networks (Computer science)

Publication Date

8-12-2023

Document Type

Dissertation

Student Type

Graduate

Degree Name

Imaging Science (Ph.D.)

Department, Program, or Center

Chester F. Carlson Center for Imaging Science (COS)

Advisor

David Messinger

Advisor/Committee Member

George Thurston

Advisor/Committee Member

Carl Salvaggio

Campus

RIT – Main Campus

Plan Codes

IMGS-PHD

Download

Share

COinS