Abstract

This thesis aims at visualizing deep convolutional neural network interpretations for aerial imagery and understanding how these interpretations change when network weights are damaged. We focus our investigation on networks for aerial imagery, as these may be prone to damages due to harsh operating conditions and are usually inaccessible for regular maintenance once deployed. We simulate damages by zeroing network weights at different levels of the network and analyze their effects on activation maps. Then we re-train the network to observe if it can recover the lost interpretations.

Visualizing changes in the neural network's interpretation, when the undamaged weights are retrained, allows us to visually assess the resiliency of a network.

Our experiments on the AID and the UC Merced Land Use aerial datasets demonstrate the emergence of object and texture detectors in convolutional networks commonly used for classification.

We further analyze these interpretations when the network is trained on one dataset and tested on another to demonstrate the robustness of feature learning across aerial datasets. We also explore the shift in interpretations when transfer learning is performed from an aerial dataset (AID) to a generic object dataset (MS-COCO).

These results illustrate how transfer learning benefits the network's internal representations. Additionally, we explore the effects of various kinds of pooling operations for class activation map extraction and their resiliency to coefficient damages.

Finally, we investigate the effects of network retraining by visualizing the change in the network's degraded interpretations before and after retraining.

Our visualization results offer insights on the resiliency of some of the most commonly used networks, such as VGG16, ResNet50, and DenseNet121.

This type of analysis can help guide prudent choices when it comes to selecting the network architecture during development and deployment under challenging conditions.

Library of Congress Subject Headings

Remote-sensing images--Data processing; Aerial photography--Data processing; Neural networks (Computer science); Convolutions (Mathematics); Fault-tolerant computing

Publication Date

12-2018

Document Type

Thesis

Student Type

Graduate

Degree Name

Computer Engineering (MS)

Department, Program, or Center

Computer Engineering (KGCOE)

Advisor

Andreas Savakis

Advisor/Committee Member

Dhireesha Kudithipudi

Advisor/Committee Member

Andres Kwasinski

Campus

RIT – Main Campus

Plan Codes

CMPE-MS

Download

Share

COinS