19 results
Now showing1 - 10 of 19
- Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Computational models of the emergence of self-exploration in 2-month-old infants(IEEE, 2025-09-16); ; ; ; Infants actively explore the relationship between actions and their associated effects (i.e., sensorimotor contingencies) before full-blown agency emerges. While there is experimental evidence for this development during the first year of life, the interplay of the associated cognitive processes is not yet well understood. This paper uses computational modeling to examine how exploratory behavior develops, based on one of the earliest experiments showing such behavior. In a seminal study of Rochat & Striano (1999), 2-month-old infants, contrary to newborns, showed differential behavioral patterns towards mouth-contingent sounds versus random sounds. This is interpreted as early evidence for action-effect exploration. We consider seven potential developmental factors as possibly explaining the emergence of active exploratory behavior in 2-month-olds: i) outcome prediction, ii) novelty preference, iii) fatigue, iv) strength, v) memory, vi) sensory noise, and vii) motor noise. These factors were implemented in both a supervised-learning model and a reinforcement learning model. Results from both models indicate that increased memory capacity with age is a key developmental factor underlying active exploration and, possibly, agency. Our code is published at: https://github.com/SpisakJ/Computational-models-of-the-emergence-of-self-exploration-in-2-month-old-infantsPublicationtype: Conference Paper29 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Final report of the project Learning Conversational Action Repair for Intelligent Robots (LeCAREbot)(2025-08); What are the principal mechanisms required to capture the robustness and interactivity of hu-man communication, given the situational, noisy and often ambiguous nature of natural lan-guage? And how, and to what extent, can we integrate these mechanisms within an embodied functional model that is computationally and empirically verifiable? We addressed these re-search questions by investigating the linguistic phenomenon of conversational repair (CR) -- a method to edit and re-interpret previously uttered sentences that were not correctly understood by the hearer....Publicationtype: Research Report49 34 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , NICOL : a Neuro-Inspired Collaborative semi-humanoid robot that bridges social interaction and reliable manipulation(IEEE, 2023-11-01); ; ; 19 148 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Grounding hindsight instructions in multi-goal reinforcement learning for roboticsThis paper focuses on robotic reinforcement learning with sparse rewards for natural language goal representations. An open problem is the sample-inefficiency that stems from the compositionality of natural language, and from the grounding of language in sensory data and actions. We address these issues with three contributions. We first present a mechanism for hindsight instruction replay utilizing expert feedback. Second, we propose a seq2seq model to generate linguistic hindsight instructions. Finally, we present a novel class of language-focused learning tasks. We show that hindsight instructions improve the learning performance, as expected. In addition, we also provide an unexpected result: We show that the learning performance of our agent can be improved by one third if, in a sense, the agent learns to talk to itself in a self-supervised manner. We achieve this by learning to generate linguistic instructions that would have been appropriate as a natural language goal for an originally unintended behavior. Our results indicate that the performance gain increases with the task-complexity.Publicationtype: Conference Paper33 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Intelligent problem-solving as integrated hierarchical reinforcement learningAccording to cognitive psychology and related disciplines, the development of complex problem-solving behaviour in biologi-cal agents depends on hierarchical cognitive mechanisms. Hierarchical reinforcement learning is a promising computational approach that may eventually yield comparable problem-solving behaviour in artificial agents and robots. However, so far, the problem-solving abilities of many human and non-human animals are clearly superior to those of artificial systems. Here we propose steps to integrate biologically inspired hierarchical mechanisms to enable advanced problem-solving skills in artificial agents. We first review the literature in cognitive psychology to highlight the importance of compositional abstraction and predictive processing. Then we relate the gained insights with contemporary hierarchical reinforcement learning methods. Interestingly, our results suggest that all identified cognitive mechanisms have been implemented individually in isolated computational architectures, raising the question of why there exists no single unifying architecture that integrates them. As our final contribution, we address this question by providing an integrative perspective on the computational challenges to develop such a unifying architecture. We expect our results to guide the development of more sophisticated cognitively inspired hierarchical machine learning architectures.Publicationtype: Journal Article157 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , The Embodied Crossmodal Self Forms Language and Interaction: A Computational Cognitive Review(2021-08-16); ; ; ; Human language is inherently embodied and grounded in sensorimotor representations of the self and the world around it. This suggests that the body schema and ideomotor action-effect associations play an important role in language understanding, language generation, and verbal/physical interaction with others. There are computational models that focus purely on non-verbal interaction between humans and robots, and there are computational models for dialog systems that focus only on verbal interaction. However, there is a lack of research that integrates these approaches. We hypothesize that the development of computational models of the self is very appropriate for considering joint verbal and physical interaction. Therefore, they provide the substantial potential to foster the psychological and cognitive understanding of language grounding, and they have significant potential to improve human-robot interaction methods and applications. This review is a first step toward developing models of the self that integrate verbal and non-verbal communication. To this end, we first analyze the relevant findings and mechanisms for language grounding in the psychological and cognitive literature on ideomotor theory. Second, we identify the existing computational methods that implement physical decision-making and verbal interaction. As a result, we outline how the current computational methods can be used to create advanced computational interaction models that integrate language grounding with body schemas and self-representations.Publicationtype: Journal Article24 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Sensorimotor Representation Learning for an “Active Self” in Robots: A Model SurveySafe human-robot interactions require robots to be able to learn how to behave appropriately in spaces populated by people and thus to cope with the challenges posed by our dynamic and unstructured environment, rather than being provided a rigid set of rules for operations. In humans, these capabilities are thought to be related to our ability to perceive our body in space, sensing the location of our limbs during movement, being aware of other objects and agents, and controlling our body parts to interact with them intentionally. Toward the next generation of robots with bio-inspired capacities, in this paper, we first review the developmental processes of underlying mechanisms of these abilities: The sensory representations of body schema, peripersonal space, and the active self in humans. Second, we provide a survey of robotics models of these sensory representations and robotics models of the self; and we compare these models with the human counterparts. Finally, we analyze what is missing from these robotics models and propose a theoretical computational framework, which aims to allow the emergence of the sense of self in artificial agents by developing sensory representations through self-exploration.Publicationtype: Journal Article21 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Developmental Robotics and its Role Towards Artificial General Intelligence(2021-03); ; ; Publicationtype: Editorial26 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Curious Hierarchical Actor-Critic Reinforcement Learning(2020-09); ; ; Hierarchical abstraction and curiosity-driven exploration are two common paradigms in current reinforcement learning approaches to break down difficult problems into a sequence of simpler ones and to overcome reward sparsity. However, there is a lack of approaches that combine these paradigms, and it is currently unknown whether curiosity also helps to perform the hierarchical abstraction. As a novelty and scientific contribution, we tackle this issue and develop a method that combines hierarchical reinforcement learning with curiosity. Herein, we extend a contemporary hierarchical actor-critic approach with a forward model to develop a hierarchical notion of curiosity. We demonstrate in several continuous-space environments that curiosity can more than double the learning performance and success rates for most of the investigated benchmarking problems. We also provide our source code (https://github.com/knowledgetechnologyuhh/goalconditionedRLbaselines) and a supplementary video (https://www2.informatik.uni-hamburg.de/wtm/videos/chacᵢcannᵣoeder₂020.mp4).Publicationtype: Conference Paper30 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Expectation Learning for Stimulus Prediction Across Modalities Improves Unisensory Classification(2019-12-11); ; ; ; Expectation learning is a unsupervised learning process which uses multisensory bindings to enhance unisensory perception. For instance, as humans, we learn to associate a barking sound with the visual appearance of a dog, and we continuously fine-tune this association over time, as we learn, e.g., to associate high-pitched barking with small dogs. In this work, we address the problem of developing a computational model that addresses important properties of expectation learning, in particular focusing on the lack of explicit external supervision other than temporal co-occurrence. To this end, we present a novel hybrid neural model based on audio-visual autoencoders and a recurrent self-organizing network for multisensory bindings that facilitate stimulus reconstructions across different sensory modalities. We refer to this mechanism as stimulus prediction across modalities and demonstrate that the proposed model is capable of learning concept bindings by evaluating it on unisensory classification tasks for audio-visual stimuli using the 43,500 Youtube videos from the animal subset of the AudioSet corpus.Publicationtype: Journal Article14