Current network data rates have made it increasingly difficult for cyber security specialists to protect the information stored on private systems. Greater throughput not only allows for higher productivity, but also creates a “larger” security hole that may allow numerous malicious applications (e.g. bots) to enter a private network. Software-based intrusion detection/prevention systems are not fast enough for the massive amounts of traffic found on 1 Gb/s and 10 Gb/s networks to be fully effective. Consequently, businesses accept more risk and are forced to make a conscious trade-off between threat and performance. A solution that can handle a much broader view of large-scale, high-speed systems will allow us to increase maximum throughput and network productivity. This paper describes a novel method of solving this problem by joining a pre-existing signature-based intrusion prevention system with an anomaly-based botnet detection algorithm in a hybrid hardware/software implementation. Our contributions include the addition of an anomaly detection engine to a pre-existing signature detection engine in hardware. This hybrid system is capable of processing full-duplex 10 Gb/s traffic in real-time with no packet loss. The behavior-based algorithm and user interface are customizable. This research has also led to improvements of the vendor supplied signal and programming interface specifications which we have made readily available.

Library of Congress Subject Headings

Computer networks--Security measures; Computer security; Computer algorithms--Design

Publication Date


Document Type


Department, Program, or Center

Computer Science (GCCIS)


Kwon, James

Advisor/Committee Member

Reynolds, Carl

Advisor/Committee Member

Tolendino, Lawrence


Note: imported from RIT’s Digital Media Library running on DSpace to RIT Scholar Works. Physical copy available through RIT's The Wallace Library at: TK5105.59 .D66 2007


RIT – Main Campus