Emily Mower Provost receives NSF CAREER Award to develop emotion and mood recognition for mental health monitoring and treatment

Prof. Emily Mower Provost has been awarded an NSF CAREER grant for her research project, “Automatic Speech-Based Longitudinal Emotion and Mood Recognition for Mental Health Monitoring and Treatment.”

Prof. Mower Provost’s research interests are in human-centered speech and video processing, multimodal interfaces design, and speech-based assistive technology. The goals of her research are motivated by the complexities of human emotion expression and perception.

Effective treatment and monitoring for individuals with mental health disorders is an enduring societal challenge. Regular monitoring increases access to preventative treatment, but is often cost prohibitive or infeasible given high demands placed on health care providers. Yet, it is critical for individuals with Bipolar Disorder (BPD), a chronic psychiatric illness characterized by mood transitions between healthy and pathological states. Transitions into pathological states are associated with profound disruptions in personal, social, vocational functioning, and emotion regulation.

Under this grant, Prof. Mower Provost will investigate new approaches in speech-based mood monitoring by taking advantage of the link between speech, emotion, and mood. The approach includes processing data with short-term variation (speech), estimating mid-term variation (emotion), and then using patterns in emotion to recognize long-term variation (mood).

The research investigates methods to estimate mood from longitudinal speech data. Current methods estimate mood from speech acoustics directly. These approaches are not sufficiently accurate for use on speech collected from an individual’s daily life. The proposed work introduces emotion as an intermediary step to simplify the mapping between speech and mood, predicated on the knowledge that emotion dysregulation is a common symptom in bipolar disorder. The proposal advances techniques to improve the robustness and generalizability of emotion recognition algorithms, resulting in low-dimensional secondary features whose variations are due to emotion. These features will be segmented and classified to provide a means to map between speech (a quickly varying signal) and user state (a slowly varying signal), advancing the state-of-the-art. The results provide quantitative insight into the relationship between emotion variation and user state variation, providing new directions and links between the fields of emotion recognition and assistive technology. The focus on modeling emotional data using time series techniques results in breakthroughs in the design of emotion recognition and assistive technology algorithms. The approaches generalize to conditions whose symptoms include atypical emotion, such as post-traumatic stress disorder, anxiety, depression, and stress.

More information about the project is available in Prof. Mower Provost’s CAREER Award Posting by NSF.

Prof. Emily Mower Provost received her PhD in Electrical Engineering from the University of Southern California (USC), Los Angeles, CA in 2010. She has been awarded a National Science Foundation Graduate Research Fellowship, the Herbert Kunzel Engineering Fellowship from USC, an Intel Research Fellowship, the Achievement Rewards For College Scientists (ARCS) Award, and the UM Oscar Stern Award for Depression Research. She is a member of Tau-Beta-Pi, Eta-Kappa-Nu, and a member of ACM, IEEE, and ISCA.

About the NSF CAREER Award

The CAREER grant is one of the National Science Foundation’s most prestigious awards, conferred for “the early career-development activities of those teacher-scholars who most effectively integrate research and education within the context of the mission of their organization.”