Perceiving Action in Space-Time: Computational and Human Perspectives

Humans are highly articulated, which leads to complex and idiosyncratic actions in space-time. This complexity has challenged computational models of human action for some time now, and yet humans themselves are highly adept at parsing action. In this talk, I will motivate the challenge of interpreting human action from spatial, temporal, and spatiotemporal points of view. Then, I will present both computational and human perspectives on modeling action. First, I will describe how video can be decomposed into a multilevel semantic scale-space using a Markov approximation framework. Within this semantic scale-space, we have conducted a visual psychophysical study of how humans perceive action, and I will report our findings in that study.

Second, I will present a computational model for human action. The method, called Action Bank, creates a high-level action space that is spanned by individual space-time actions. Query videos are projected into this action space and non-linear classifiers are learned for recognition. Experiments demonstrate how space-time, action-specific modeling can outperform conventional feature-based methods that do not leverage space-time continuity. Third, I will bring these two perspectives together in a "full-circle" experiment leveraging ideas from both camps. Time-permitting, I will relate these findings to other work in my group in computational neuroscience and cognitive robotics.
Corso is an associate professor of Computer Science and Engineering at SUNY Buffalo. He received his Ph.D. at The Johns Hopkins University in 2006 (from the Computational Interaction with Physical Systems Lab), the M.S.E Degree from The Johns Hopkins University in 2002 and the B.S. Degree with honors from Loyola College In Maryland. He spent two years as a post-doctoral fellow at the University of California, Los Angeles affiliated with Medical Imaging Informatics, the Laboratory of Neuro Imaging, and the Center for Image and Vision Science. He is the recipient of the Army Research Office Young Investigator Award 2010 (robotics), NSF CAREER award 2009 (computer vision), SUNY Buffalo Young Investigator Award 2011, a member of the 2009 DARPA Computer Science Study Group (data mining), and a recipient of the Link Foundation Fellowship in Advanced Simulation and Training (physically-grounded vision). Corso has authored more than ninety peer-reviewed papers on topics of his research interest including computer vision, robotics, data science, and medical imaging. He is a member of the AAAI, IEEE and the ACM. He is PI on more than $5 million in research funding from major federal agencies, including NSF, NIH, DARPA, ARO, and IARPA.