Abstract

Anomaly detection has attracted increasing attentions from diverse domains, including medicine, public safety, and military operations. Despite its wide applicability, anomaly detection is inherently challenging as abnormal activities are usually rare and unbounded in nature. This makes it difficult and expensive to identify all potential anomalous events and label them during the training phase so that a detection model can provide robust prediction on unseen event data. Unsupervised and semi-supervised learning models have been explored, which achieve a decent detection performance with no or much less annotations. However, these models can be highly sensitive to outliers (\ie normal samples that look different from other normal ones) or multimodal scenarios (\ie existence of multiple types of anomalies), leading to much worse detection performance under these situations. The imbalanced class distribution of the normal data samples poses a further challenge as the model may be confused between the normal samples from a minority class and true anomalies. To systematically address the key challenges outlined above, this dissertation contributes the first Robust Weakly Supervised Learning (RWSL) framework that provides fundamental support for real-world anomaly detection using only weak and/or sparse learning signals. The proposed RWSL framework offers a principled learning paradigm to deal with the rare and unbounded nature of real-world anomalies, which allows a statistical learning model to be robustly trained using only high-level supervised signals while generalizing well in few-shot settings. The framework is comprised of three interconnected components. The first research component integrates Robust Distributionally Optimization (DRO) with Bayesian learning, leading to a novel Bayesian DRO model that achieves robust detection performance using weak learning signals coupled with outliers and multimodal anomalies. The Bayesian DRO model is further augmented with non-parametric submodular optimization and active instance sampling to improve both the reliability and accuracy of the detection performance. The second research component leverages the evidential theory and its fine-grained uncertainty formulation to tackle anomaly detection coupled with imbalanced class distribution of normal data samples. An adaptive Distributionally Robust Evidential Optimization (DREO) training process is developed to boost the anomaly detection performance by accurately differentiating minority class samples and true anomalies using evidential uncertainty. Evidential learning is further integrated with a transformer architecture, leading to an Evidential Meta Transformer (MET) for reliable anomaly detection in the few-short setting. Finally, the third research component aims to ensure an unbiased (\ie fair) and better-calibrated model with improved anomaly detection performance by avoiding the overconfidence predictions stemming from the memorization effect seen in deep neural networks. To achieve this, the Distributionally Robust Ensemble (DRE) is proposed that learns multiple diverse and complementary sparse sub-networks through the utilization of DRO properties. By facilitating these sparse sub-networks to capture different data distributions across varying levels of complexity, they naturally complement each other resulting in improved model calibration with enhanced anomaly detection capability.

Library of Congress Subject Headings

Supervised learning (Machine learning); Anomaly detection (Computer security); Bayesian statistical decision theory; Mathematical optimization

Publication Date

9-2023

Document Type

Dissertation

Student Type

Graduate

Degree Name

Computing and Information Sciences (Ph.D.)

Department, Program, or Center

Computing and Information Sciences Ph.D, Department of

College

Golisano College of Computing and Information Sciences

Advisor

Qi Yu

Advisor/Committee Member

Zhiqiang Tao

Advisor/Committee Member

Rui Li

Campus

RIT – Main Campus

Plan Codes

COMPIS-PHD

Download

Share

COinS