Statistics and Computation in the Age of Massive Data

Abstract – There are many issues remaining to be addressed, or even formulated, at the interface of statistics and computation. One way to capture the current state of affairs is the following: If we view data as a resource, how can it be that in many practical problems of interest we find ourselves embarrassed by being given too much data? Our inferential procedures typically use polynomial amounts of time and space but that doesn’t suffice; we need to be able to guarantee that on a fixed computational budget the statistical risk decreases as the number of data points grows (without bound). A general theory not yet being available, in this talk I present three vignettes that describe various lines of attack on the problem: one involving the bootstrap, another involving matrix completion algorithms and the third involving phylogenetic analysis in the regime of large numbers of taxa. All three vignettes involve divide-and-conquer strategies, with the third vignette being particularly interesting in this regard (divide-and-conquer arises from Poisson thinning). [Joint work with Alexandre Bouchard-Cote, Ariel Kleiner, Lester Mackey, Purna Sarkar and Ameet Talwalkar.]

Biography – Michael I. Jordan is the Pehong Chen Distinguished Professor in the Department of Electrical Engineering and Computer Science and the Department of Statistics at the University of California, Berkeley. He received his Masters in Mathematics from Arizona State University, and earned his PhD in Cognitive Science in 1985 from the University of California, San Diego. He was a professor at MIT from 1988 to 1998. His research in recent years has focused on Bayesian nonparametric analysis, probabilistic graphical models, spectral methods, variational methods, kernel machines and applications to problems in statistical genetics, signal processing, computational biology, information retrieval and natural language processing. Prof. Jordan is a member of the National Academy of Sciences, a member of the National Academy of Engineering and a member of the American Academy of Arts and Sciences. He is a Fellow of the American Association for the Advancement of Science. He has been named a Neyman Lecturer and a Medallion Lecturer by the Institute of Mathematical Statistics. He is a Fellow of the ACM, the CSS, the IMS, the IEEE, the AAAI and the ASA.