Putting Your Database on Autopilot: Self-driving Database Management Systems

Database management systems (DBMSs) are essential for modern data-driven applications. However, they are notoriously difficult to deploy and administer because they have many aspects that one can change that affect their performance, including database physical design and system configuration. There are existing methods that recommend how to change these aspects of databases for an application. But most of them still require humans to make final decisions on what changes to apply and when to apply them. Thus, DBMS administrations today remain onerous and costly.

In this talk, I present a self-driving DBMS architecture that enables automatic system management and removes the administration impediments. Our approach consists of three frameworks inspired by self-driving car architectures: (1) workload forecasting, (2) behavior modeling, and (3) action planning. The workload forecasting framework predicts the query arrival rates under varying database workload patterns using an ensemble of time-series forecasting models. The behavior modeling framework constructs fine-grained machine learning models that predict the runtime behavior of the DBMS. Lastly, the action planning framework generates a sequence of optimization actions based on these forecasted workload patterns and behavior model estimations. It uses receding horizon control and Monte Carlo tree search to effectively approximate the complex optimization problem.

Our forecasting-modeling-planning architecture enables an autonomous DBMS that proactively plans for optimization actions without expensive testing. It automatically applies the actions at proper times, holistically controls all system aspects, and provides explanations on its decisions.