Deep Reinforcement Learning for Sequential Decision Making Tasks with Natural Language Interaction

The success of Deep Learning (DL) on visual perception has led to rapid progress on Reinforcement Learning (RL) tasks with visual inputs. More recently, Deep Learning is showing promise at certain kinds of supervised natural language problems and this too is making its way into helping on RL tasks with natural language inputs. In this talk, I will describe two projects in this direction from my group. The first (url 1 below) involves learning to query, reason, and answer questions on simple forms of ambiguous texts designed to focus on a specific problem that occurs in dialog systems. The second (url 2 below) involves zero shot generalization to unseen instructions in a 3d maze navigation task for which we develop a hierarchical DeepRL architecture.