The KiMeKo partners have acquired many skills in previous KI-Med projects - in particular KI-SIGS funded by the BMWK and KI-Lab funded by the BMBF - but these are decentralized, unsystematized and in need of supplementation. This is precisely the aim of the five KIMKO sub-projects: to bundle and deepen expertise in such a way that the KI-Med ecosystem can offer standardized services, i.e. methods, tools and workflows, as far as possible. On this basis, the AI-Med collaboration platform is intended to ensure that, on the one hand, the collaboration between medical professionals, IT specialists and companies (the stakeholder groups), which is essential for AI-Med applications, is made significantly more efficient (faster and qualitatively improved); in order to implement larger - possibly also disruptive - strategic AI-Med applications with truly broad usage potential in healthcare with a higher chance of success. And on the other hand, the aim is to accelerate the development of such AI-Med innovations to such an extent that AI-Med medical products can actually be developed with regulatory approval, which alone find their way into healthcare and thus enable economic value creation in the healthcare system and for companies. The five sub-projects are all being carried out by the universities' IT institutes with the advisory involvement of clinics and companies, as the intention is not yet to develop AI-Med applications as such, but to create the strong conditions that will enable them to scale in the context of the AI-Med ecosystem in the future.

Conversational natural language is subject to noise, incompletions and grammatically ambiguous phrasing. To increase the robustness of communication, human conversation partners typically build on conversational repair (CR) to iteratively and interactively resolve misunderstandings. In the context of human-robot interaction, CR provides the possibility to interrupt and to repair a misunderstood instruction that is already being executed. However, current approaches do not consider the conversational repair of misunderstandings in human-robot dialog, even though this would significantly increase the robustness of human-robot interaction. The goal of this project is to fill this gap by addressing two core problems that have hindered existing approaches to successfully address conversational action repair for human-robot interaction. The first problem is the realization of an adaptive context-specific state model that integrates language with action. Most dialog systems consider only verbal communication, and they ignore that human communication is an embodied multi-modal process that is grounded in physical interaction. So how can we realize a scalable model that considers situated conceptual state representations for mixed verbal-physical interaction? To address this first problem, this project builds on a neuro-symbolic approach that integrates our previous work on embodied semantic parsing with our expertise in deep reinforcement learning. Herein, we will research a hybrid data- and knowledge-driven model for compositional interaction states that link the physical world state with semantics in language and dialog.The second problem pertains to the noise, disfluency, and polysemy of spoken natural language. Existing learning-based parsers are robust enough to parse noisy spoken language but they require large amounts of training data. So how can we realize a robust semantic parser that is data efficient while considering the mixed verbal-physical interaction? To address this second problem, this project complements our previous semantic parsing methods with a neural machine-translation approach. To this end, we will exploit the reward signal of the reinforcement learning as an additional data source to improve the data efficiency of the neural parser. The data required for this project will be generated using crowdsourcing, and the evaluation will be conducted on a humanoid robot. We expect the project to generate impact as a new approach for human-robot interaction, and to contribute novel methods for representation learning to the scientific communities in the fields of natural language understanding, machine learning, and intelligent robotics.

In this project MoReSpace, we will investigate the extent to which the transfer of learning is responsible for the development of a "self", and hypothesize that a conflict-driven attention model plays a major role. In the first part of our project, we investigate the transfer of previously learned action-effect associations to new unexpected environmental dynamics. Here, we put a strong focus on cognitive plausibility and motivate our model with psychological phenomena such as "haptic neglect". The phenomenon occurs, for example, when the computer mouse is inverted and the mouse pointer is directed in the opposite direction in each case. In such scenarios, psychologists have found reduced perception of the haptic and proprioceptive senses. Our hypothesis is that this is due to a conflict-driven attention mechanism that improves the ability to deal with such new dynamics. We will evaluate our model on a physical robot, and we will theoretically substantiate it with our collaboration partners from psychology. In the second part of the project, we will focus on imitation learning. Our hypothesis is that the attention model captures some psychological properties that are important for the human ability to change perspective and to imitate. We hypothesize that this will lead to novel methods of imitation learning for robots. We expect these methods to lead to significant improvements in the learning performance. We will evaluate this empirically and reproducibly.

The general aim of the project is to reveal necessary conditions for the emergence of internal representations associated with the self, when dealing with truly embodied agents. This will be based on the study of Helmholtz machines that implement prediction and recognition as prerequisite for optimal control. Furthermore, the aim is to study to what extent these processes generate high integrated information in the sense of Tononi's Integrated Information Theory (IIT) of consciousness. This will provide insights about the mechanisms that underly the phenomenal self. Based on information theory, which is quantitative in nature, we expect to identify transitions between qualitatively different kinds of embodiments, thereby relating our work to Metzinger's orders of embodiment. In this period of the DFG SPP "The Active Self”, a hierarchy of controller architectures with increasing granularity will be developed, ultimately leading to neuronal architectures. Corresponding learning algorithms from the theory of Helmholtz machines, versions of the wake-sleep algorithm, suggest a close connection to the Free Energy Principle, which will provide a conceptual and formal basis for the project. Based on controller architectures with various granularities, the project will analyse corresponding information flows in sensorimotor loops of robotic systems, in collaboration with Verena V. Hafner's group. The aim is to verify the increase of information integration when learning is involved and incorporates a forward model for prediction and an inverse model for control.

Embodied systems are subject to their sensorimotor constrains, which shape the way these systems can perceive and act on the external world. Corresponding distinctions in the world constitute the Umwelt of such systems, a notion that has been introduced by Uexküll for the first time in 1926 who also studied it in the context of various animals. In order to control beneficial behavior, embodied systems have to structure some aspects of their Umwelt in terms of an internal model. We distinguish various degrees of this structure, which is, for particular conditions, correlated with conscious experience and a self-representation. Tononi's integrated information theory of consciousness can provide indicators for this. The aim of the project is to extend and develop information-theoretic methods that will allow us to use the integrated information theory also in the context of embodied systems. In particular, the project will reveal the requirements for the emergence of a self-representation through the interaction of the system with the world. The methods will be demonstrated and exemplified in terms of robotics experiments.

In this project, we will investigate how far the transfer of learning is responsible for the development of a “self”. We will, therefore, present a computational ideomotor approach, and hypothesize that the transfer is possible due to a hierarchical structure of action-effect associations, such that training a specific narrow low-level task indirectly trains higher cognitive skills that are involved in other low-level tasks. For example, manipulating objects and balancing are two low-level tasks that both involve the cognitive skill of mental rotation. Consequently, according to our hypothesis, the mental rotation skill will benefit from balance training, which in turn also triggers an improvement of the learning of the grasping task. We will address our hypothesis by implementing a computational and neurocognitively plausible neural network architecture evaluated on a physical humanoid robot. Our expected contribution is a functional neurocognitively plausible deep reinforcement neural network model for ideomotor transfer learning that is verifiable on a reproducible robotic platform.