The Analysis, Modeling and Detection of Botnet-based Hosting Services and Future Threats

Botnets"”vast collections of compromised computers (i.e., bots) under the control of a
botmaster"”have become one of the greater threats facing the Internet community due
to their versatility and financial appeal. Much of their success, financial and otherwise,
can be attributed to 4 properties/strategies: stealth, first and foremost, bots want
to remain stealthy in their infection and occupation, keeping the botnet resources
high; modularity, allows an already infected machine to update its bot malware,
granting it new functionality; Command and Control, permitting coordination and
post-deployment modification of the botnet functionality and behavior as needed for
various scams or to evade detection; and content delivery mechanisms, such as botnet-
based hosting services and fast-flux (FF) DNS strategies, permit botmasters to serve
scams and malicious content to victims for profit or the purpose or swelling their
botnet ranks. Throughout the dissertation, we study this stealthy aspect of botnets
and its imposed limitations, exploring botnets' primary content delivery mechanism"”
botnet-based hosting services utilizing FF DNS advertising strategies"”and the future
mobile botnet threatscape emerging with the increase in mobile devices and wireless connectivity.

This dissertation makes four primary contributions. First, it introduces an automated enterprise solution, called RB-Seeker, for quickly and accurately detecting
domains and bots involved in botnet-based hosting services. Analyzing spam emails
from multiple sources and NetFlow traces gathered from the core network router, it
identifies domains utilizing redirection, which are then monitored by a DNS probing
engine to identify botnets. The feasibility of RB-Seeker as an automatic and accurate
botnet detection system for enterprise networks has been demonstrated by evaluating
it on a large university network. Second, the dissertation grants insight into the global
advertising strategy, capabilities, and limitations of botnet-based hosting services by
deploying DIGGER"”a distributed DNS-monitoring system comprising hundreds of
nodes spanning multiple continents. For an extended period of time, DIGGER monitors the DNS-advertising behavior and online connectivity for a set of suspicious
domains, which is continuously updated from spam emails and online repositories
of malicious domains. Analyzing these DNS results, we are able to determine the
current DNS-advertising strategies employed by botnet-based hosting services and
identify powerful, intrinsic behavioral features for use in detection. Third, this dissertation analyzes the effectiveness of state-of-art FF detectors, demonstrating how
they can be thwarted with current botnet resources by mimicking benign domains. It
also evaluate mimicry attacks against its novel spatial-detection system and introduce
a new detection metric, percent connectivity, that helps defend against mimicry attacks. Based on realistic assumptions inferred from DIGGER's empirically observed
trends, the dissertation presents formal models for bot decay, online availability, DNS-
advertisement strategies and performance, demonstrating the effectiveness of different
mimicry attacks in evading detection systems and evaluating their effects on the overall online availability and capacity of botnets. Finally, the dissertation looks to the
future of botnets on the rapidly advancing mobile market"”alluring due to their high mobility, multiple communication channels and always-on connectivity. It evaluates
how successfully a mobile botnet utilizing only open WiFi networks could receive
commands and issue attacks, and how performance is improved through intelligent
AP-selection exploiting predictable routes, such as those of buses. Malicious behavior could be spread across many different open WiFi networks (obtaining a new IP
at each network) and hidden amongst a plethora of benign traffic. Using real-world
WiFi network locations, mobility traces and bus routes for the city of San Francisco,
we design and simulate an open WiFi-based mobile botnet, demonstrating that it can
pose a serious threat and provide an ideal mechanism for botmasters transitioning to
the mobile landscape.