Extracting and Querying Probabilistic Information in BayesStore

In the past few years, the number of applications that need to process large-scale data has grown remarkably. The data driving these applications is often uncertain, as is the analysis, which often involves probabilistic modeling and inference. Examples include sensor-based monitoring, information extraction and online advertising. Prior to our work, probabilistic database research advocated an approach in which uncertainty is modeled by attaching probabilities to data items. However, such systems do not and cannot take advantage of the wealth of Statistical Machine Learning (SML) research, because they are unable to represent and reason about the pervasive probabilistic correlations in the data.

In my thesis, I proposed, built, and evaluated BayesStore, a probabilistic database system that natively supports SML models and various inference algorithms to perform advanced data analysis. This marriage of database and SML technologies creates a declarative and efficient probabilistic processing framework for applications dealing with large-scale uncertain data. I have explored a variety of research challenges, including extending the database data model with probabilistic data and statistical models, defining relational operators (e.g., select, project, join) over probabilistic data and models, developing joint optimization of inference operators and the relational algebra, and devising novel query execution plans. I used information extraction over text as the driving application. My research shows that using in-database SML methods to extract and query probabilistic information can significantly improve answer quality. Moreover, it shows that optimizations for query-driven SML inference lead to orders-of-magnitude speed-up on large corpora.

Daisy Zhe Wang is a Ph.D. student in Computer Science department at UC Berkeley. She is a member of the database research group, RAD Lab and AMP Lab. She received her B.A.Sc. degree with honors from University of Toronto in 2005. Her research focuses on data management systems that support scalable, declarative, on-line data analytics based on Statistical Machine Learning models. She collaborated with IBM Research at Almaden, AT&T Research, Yahoo! Research, and Intel Research on probabilistic information management. She also had industry experience at Google Inc. and IBM Toronto Lab.