Efficient Belief Propagation for Perception and Manipulation in Clutter

Link: https://bluejeans.com/5658255708

Autonomous service robots are required to perform tasks in common human indoor environ- ments. To achieve goals associated with these tasks, the robot should continually perceive, reason and plan to manipulate objects, which we term as goal-directed manipulation. Perception remains the most challenging of all stages, as common indoor environments typically pose problems in recognizing and localizing objects under inherent occlusions with physical interactions resulting in partial observations. Despite recent progress in the field of robot perception, accommodating perceptual uncertainty due to partial observations remains challenging and needs to be addressed to achieve the desired autonomy.

In this dissertation, we address the problem of perception under uncertainty for robot manip- ulation in cluttered environments using generative inference methods. Specifically, we aim to enable robots to perceive partially observable environments by maintaining an approximate probability distribution as a belief over possible scene hypothesis. This belief representation captures uncertainty in object poses resulting from inter-object occlusions and physical interactions. The research efforts presented in this thesis are towards developing appropriate state representations and inference techniques to maintain such belief over contextually plausible scene states. We focus on addressing challenges involved in: 1) generating and maintaining plausible scene hypotheses, 2) reducing the inference search space that typically grows with respect to the number of objects in a scene, 3) preserving scene hypotheses over continuous observations.

In order to generate and maintain plausible scene hypotheses, we propose a physics informed scene estimation method that combines Newtonian physics engine within a particle based Bayesian inference framework. To scale up to accommodate a higher number of objects, we describe factorization of scene state into objects and their parts to perform collaborative particle based inference. This resulted in Pull Message Passing for Nonparametric Belief Propagation (PMPNBP) algorithm that caters to the demands of the high-dimensional multimodal nature of cluttered scenes while achieving computational tractability.  Towards extending our PMPNBP algorithm over continuous observations, we explore hypotheses augmentation techniques to inform and maintain belief over time, and demonstrate its applicability on tracking articulated objects under severe occlusions.