Grounding hindsight instructions in multi-goal reinforcement learning for robotics

Publikationstyp
Conference Paper
Date Issued
2022-09
Sprache
English
Author(s)
Röder, Frank  
Eppe, Manfred  
Wermter, Stefan  
Institut
Data Science Foundations E-21  
TORE-URI
http://hdl.handle.net/11420/14360
Start Page
170
End Page
177
Citation
2022 IEEE International Conference on Development and Learning (ICDL 2022): 170-177
Contribution to Conference
IEEE International Conference on Development and Learning, ICDL 2022  
Publisher DOI
10.1109/ICDL53763.2022.9962207
Scopus ID
2-s2.0-85143403930
ArXiv ID
2204.04308
Publisher
IEEE
ISBN
978-1-6654-1311-4
978-1-6654-1312-1
978-1-6654-1310-7
Peer Reviewed
true
This paper focuses on robotic reinforcement learning with sparse rewards for natural language goal representations. An open problem is the sample-inefficiency that stems from the compositionality of natural language, and from the grounding of language in sensory data and actions. We address these issues with three contributions. We first present a mechanism for hindsight instruction replay utilizing expert feedback. Second, we propose a seq2seq model to generate linguistic hindsight instructions. Finally, we present a novel class of language-focused learning tasks. We show that hindsight instructions improve the learning performance, as expected. In addition, we also provide an unexpected result: We show that the learning performance of our agent can be improved by one third if, in a sense, the agent learns to talk to itself in a self-supervised manner. We achieve this by learning to generate linguistic instructions that would have been appropriate as a natural language goal for an originally unintended behavior. Our results indicate that the performance gain increases with the task-complexity.
Subjects
reinforcement learning
language grounding
instruction following
hindsight instruction
human-robot interaction
DDC Class
004: Informatik
600: Technik
620: Ingenieurwissenschaften