Network analysts are bombarded with large amounts of low level data, posing great challenges for them to differentiate and recognize critical multistage attacks. Multistage attacks are performed by hackers to compromise one or more machines in a network to gradually gain access to critical information or network operation hidden behind layers of firewall rules. These multistage attacks, composed of correlated Intrusion Detection System (IDS) alerts, can be diverse in the way they progress and penetrate the network. There exists no current literature defining how these diverse multistage attacks may be classified or categorized. This work aims to perform unsupervised learning to cluster and identify types of multistage attacks. Multistage attacks may attack services of different types, often indicating the behavior of attack penetration into the network. Divisive Hierarchical Clustering has been shown to effectively uncover underlying community structure of entities sharing similar features. This work investigates the use of attacked services as the feature and performs Divisive Hierarchical Clustering to identify groups of similar multistage attacks. The notion of social network analysis is leveraged to determine the optimal community structure with the highest modularity. The resulting clusters and dendrograms provide not only insights on characterizing multistage attacks, but also a means of reducing the data volume while enhancing the level of analysis. The outcomes of the proposed methodology are expected to improve situation awareness in the presence of many diverse multistage attacks.

Library of Congress Subject Headings

Computer networks--Security measures; Computer crimes--Prevention; Cluster analysis

Publication Date


Document Type


Student Type


Degree Name

Computer Engineering (MS)

Department, Program, or Center

Computer Engineering (KGCOE)


Shanchieh Jay Yang

Advisor/Committee Member

Roy Czernikowski

Advisor/Committee Member

Andres Kwasinski


RIT – Main Campus

Plan Codes