Unlocking musical creativity with generative AI

Zoom link for remote attendees: password 123123

Abstract: In this talk, I will present my work on developing and responsibly deploying generative AI systems that unlock and augment human creative potential in music. While we all possess a remarkably sophisticated intuition for and appreciation of music, conventional tools for creative musical expression (e.g., instruments, music notation) are inaccessible to those of us without formal training. To lower the barrier to entry, I develop generative AI systems with intuitive forms of control (e.g., singing) that allow users to easily translate their ideas into music. My research also aims to augment the creative potential of experts. To this end, I develop generative AI methods that can support realistic co-creation workflows for musicians, analogous to tools like Copilot for programmers. Methodologically, my work often centers around language models (LMs), and involves building new LM methods to confront the unique challenges posed by the domain of music, such as modeling long sequences and understanding multimodal relationships. Another challenge of working in creative domains is evaluation—to confront this, my work involves deploying systems to real-world users, so that we may better understand how systems help users accomplish real creative goals. More broadly, we are in the midst of a pivotal moment in music AI research, where technological developments are suddenly translating into real-world impact. Accordingly, I will discuss how I approach my current and future research goals in a responsible fashion, commensurate with the broad economic, cultural, and social importance of music.

Bio: Chris Donahue is an Assistant Professor in the Computer Science Department at CMU, and a part-time research scientist at Google DeepMind working on the Magenta project. His research goal is to develop and responsibly deploy generative AI for music and creativity, thereby unlocking and augmenting human creative potential. In practice, this involves improving machine learning methods for controllable generative modeling of music and audio, and deploying real-world interactive systems that allow anyone to harness generative music AI to accomplish their creative goals through intuitive forms of control. Chris’s research has been featured in live performances by professional musicians like The Flaming Lips, and also empowers hundreds of daily users to convert their favorite music into interactive content through his website Beat Sage. His work has also received coverage from MIT Tech Review, The Verge, Business Insider, and Pitchfork. Before CMU, Chris was a postdoctoral scholar in the CS department at Stanford advised by Percy Liang. Chris holds a PhD from UC San Diego where he was jointly advised by Miller Puckette (music) and Julian McAuley (CS).