Memory Models: A Case for Rethinking Parallel Languages and Hardware

Abstract – The era of parallel computing for the masses is here, but writing correct parallel programs remains far more difficult than writing sequential programs. Aside from a few domains, most parallel programs are written using a shared-memory approach. The memory model, which specifies the meaning of shared variables, is at the heart of this programming model. Unfortunately, it has involved a tradeoff between programmability and performance, and has arguably been one of the most challenging and contentious areas in both hardware architecture and programming language specification. Recent broad community-scale efforts have finally led to a convergence in this debate, with popular languages such as Java and C++ and most hardware vendors publishing compatible memory model specifications. Although this convergence is a dramatic improvement, it has exposed fundamental shortcomings in current popular languages and systems that prevent achieving the vision of structured and safe parallel programming.

I will discuss the path to the above convergence, the hard lessons learned, and their implications. A cornerstone of this convergence has been the view that the memory model should be a contract between the programmer and the system – if the programmer writes disciplined (data-race-free) programs, the system will provide high programmability (sequential consistency) and performance. I will discuss why this view is the best we can do with current popular languages, and why it is inadequate moving forward. I will then discuss research directions that eliminate much of the concern about the memory model, but require rethinking popular parallel languages and hardware. In particular, I will argue that parallel languages should not only promote high-level disciplined models, but they should also enforce the discipline. Further, for scalable and efficient performance, hardware should be co-designed to take advantage of and support such disciplined models. I will use the Deterministic Parallel Java language and DeNovo hardware projects at Illinois as examples of such directions. This talk draws on collaborations with many colleagues over two decades, and is mostly based on an upcoming CACM paper with Hans-J. Boehm.

Biography – Sarita Adve is Professor of Computer Science at the University of Illinois at Urbana-Champaign. Her research interests are in computer architecture and systems, parallel computing, and power and reliability-aware systems. She received the ACM SIGARCH Maurice Wilkes award in 2008, was named a University Scholar by the University of Illinois in 2004, and received an Alfred P. Sloan Research Fellowship in 1998. She received the Ph.D. in Computer Science from Wisconsin in 1993.