Understanding the 3D World from Images

In this talk I will introduce a novel paradigm for jointly addressing two fundamental problems in computer vision: 3D reconstruction and object recognition. Most of the state-of-the-art methods deal with these two tasks separately. Methods for object recognition typically describe the scene as a list of object class labels, but are unable to account for their 3D spatial organization. Most of the approaches for 3D scene modeling produce accurate metric reconstructions but are unable to infer the semantic content of their components. A major line of work from my group in recent years is to explore methodologies that seek to fill this gap and to coherently describe objects and object components while simultaneously integrating their 3D spatial arrangement in the scene's physical space as well as understanding the activities that are taking place in the scene. This research is relevant to many application areas such as autonomous or assisted navigation, robotics, augmented reality, human-computer interaction, automatic 3D modeling of urban environments and surveillance.