Socially intelligent autonomous agents that learn from human reward

In the future, autonomous agents will operate in human inhabited environments in many real world applications and become an integral part of human’s daily lives. Therefore, when autonomous agents enter into the real world, they need to adapt to many novel, dynamic and complex situations that cannot be imagined and pre-programmed in the lab and try to learn new skills. Meanwhile, human users may also want to teach agents behaviors they like. Interactive shaping is a teaching method that has been developed and proven to be a powerful technique for enabling autonomous agents to learn how to perform tasks from real time human reward signals, i.e., evaluations of the quality of an agent’s behavior delivered by a human observer. However, though the agent can already learn from such human-delivered reward signals, the agent learning critically depends on the quality and quantity of the interaction between the human teacher and the agent. In this thesis, we use interactive shaping, the TAMER framework in particular, to investigate methods for increasing the efficiency of agent learning from human reward. We consider solutions to better engage a human trainer from the perspectives of both the agent and human trainer. The work in this thesis has made several contributions towards the understanding of interactive shaping by developing methods to augment the agent’s learning through it and further ease the human teacher’s cognitive load. Our results may generalize to other task domains and apply to other interactive learning algorithms.