Diamond: A System for Brute-force Interactive Search

As storage capacities continue to increase and digital data is more
readily available, it has become increasingly difficult to find specific
items in large collections of complex data (photographs, protein
structures, etc).

This talk describes a system, Diamond, which has been designed to enable
interactive applications that search large, un-indexed data sets.
Diamond enables efficient search through "early discard" eliminating
the bulk of the irrelevant items as early as possible. Diamond provides
additional efficiency by re-ordering search tasks and by dynamically
partitioning computation between host computer and the storage devices
to adapt to different hardware configurations and network bandwidths.
This talk will present some performance measurements using our
application for searching digital images.

Larry Huston is a Senior Researcher at the Intel Research Lab in
Pittsburgh. His research interests include distributed systems,
networking, and hardware-software interactions. Dr. Huston is currently
co-PI on the Diamond project that is exploring how to enable interactive
searches of large collections of complex data. Prior to joining the
Pittsburgh Research Lab, Dr. Huston was a Principal Software Architect
for Intel's Network Processor group where he designed the programming
framework that is used for Intel's IXP family of Network processors.
Prior to joining Intel, Dr. Huston was member of the NetBoost
Corporation where he helped designed and implement NetBoost's
programming framework for accelerating network applications (firewalls,
intrusion detection, etc.) on NetBoost's proprietary hardware. Before
NetBoost, Larry was a member of the technical staff at Ipsilon networks
where he help design and implement a distributed switching system that
dynamically mapped network flows onto ATM switched circuits. Larry
received Bachelor degrees in Aerospace Engineering and Computer
Engineering from the University of Michigan in 1990, Masters Degrees in
Computer Engineering and Aerospace Engineering in 1992, and his Ph.D. in
Computer engineering from the University of Michigan in 1995. Dr.
Huston's thesis topic was access to distributed file systems in the face
of weak and intermittent network connectivity.