Robust Scene Estimation for Goal-directed Robotic Manipulation in Unstructured Environments

This defense is remote. The link to the event is : https://bluejeans.com/802904654/

To make autonomous robots taskable such that they function properly, meet human expectations, and interact fluently with human partners, they must be able to perceive and understand the semantic aspects of their environments. However, the everyday world that humans inhabit is dynamic and unstructured. Many semantic aspects of the unstructured environment are difficult for robots to sense directly due to the limited field-of-view and noisy observations from their onboard sensors. The unstructured environment (e.g. objects in dense clutter) further results in substantial perceptual aliases from state-of-the-art neural networks. Furthermore, robots cannot reply on complete knowledge from the single observation so they have to continuously acquire information necessary to support decision making. In this thesis, we propose three methods that perform robust scene estimation in unstructured environments.

We first propose Axiomatic Particle Filter (APF), a pure generative approach, to estimate the distribution over scene graphs, in which each object is a node and inter-object relations are edges. Possible world states are hypothesized to explain the true world state and the only maximal likely state estimate is used for planning.  The ambiguity can then be alleviated with further information after a robot manipulation action. However, this approach assumes known object identification which leads to known data association. To avoid this assumption, we propose Sequential Scene Understanding and Manipulation (SUM) to leverage the strength from discriminative learning and generative probabilistic method. SUM considers noisy detections from learning algorithms, performs a greedy data association and addresses this uncertainty through a generative process. To deal with objects in severe occlusions, we propose GeoFusion, a SLAM-based scene estimation method, to fuse noisy measurements from different viewpoints. Geometric consistency is considered at the object-level. It offers a fast and reliable data association process which is robust to perceptual aliases. Meanwhile it enforces proper geometric contact between objects through graph optimization.