The KiMeKo partners have acquired many skills in previous KI-Med projects - in particular KI-SIGS funded by the BMWK and KI-Lab funded by the BMBF - but these are decentralized, unsystematized and in need of supplementation. This is precisely the aim of the five KIMKO sub-projects: to bundle and deepen expertise in such a way that the KI-Med ecosystem can offer standardized services, i.e. methods, tools and workflows, as far as possible. On this basis, the AI-Med collaboration platform is intended to ensure that, on the one hand, the collaboration between medical professionals, IT specialists and companies (the stakeholder groups), which is essential for AI-Med applications, is made significantly more efficient (faster and qualitatively improved); in order to implement larger - possibly also disruptive - strategic AI-Med applications with truly broad usage potential in healthcare with a higher chance of success. And on the other hand, the aim is to accelerate the development of such AI-Med innovations to such an extent that AI-Med medical products can actually be developed with regulatory approval, which alone find their way into healthcare and thus enable economic value creation in the healthcare system and for companies. The five sub-projects are all being carried out by the universities' IT institutes with the advisory involvement of clinics and companies, as the intention is not yet to develop AI-Med applications as such, but to create the strong conditions that will enable them to scale in the context of the AI-Med ecosystem in the future.

Conversational natural language is subject to noise, incompletions and grammatically ambiguous phrasing. To increase the robustness of communication, human conversation partners typically build on conversational repair (CR) to iteratively and interactively resolve misunderstandings. In the context of human-robot interaction, CR provides the possibility to interrupt and to repair a misunderstood instruction that is already being executed. However, current approaches do not consider the conversational repair of misunderstandings in human-robot dialog, even though this would significantly increase the robustness of human-robot interaction. The goal of this project is to fill this gap by addressing two core problems that have hindered existing approaches to successfully address conversational action repair for human-robot interaction. The first problem is the realization of an adaptive context-specific state model that integrates language with action. Most dialog systems consider only verbal communication, and they ignore that human communication is an embodied multi-modal process that is grounded in physical interaction. So how can we realize a scalable model that considers situated conceptual state representations for mixed verbal-physical interaction? To address this first problem, this project builds on a neuro-symbolic approach that integrates our previous work on embodied semantic parsing with our expertise in deep reinforcement learning. Herein, we will research a hybrid data- and knowledge-driven model for compositional interaction states that link the physical world state with semantics in language and dialog.The second problem pertains to the noise, disfluency, and polysemy of spoken natural language. Existing learning-based parsers are robust enough to parse noisy spoken language but they require large amounts of training data. So how can we realize a robust semantic parser that is data efficient while considering the mixed verbal-physical interaction? To address this second problem, this project complements our previous semantic parsing methods with a neural machine-translation approach. To this end, we will exploit the reward signal of the reinforcement learning as an additional data source to improve the data efficiency of the neural parser. The data required for this project will be generated using crowdsourcing, and the evaluation will be conducted on a humanoid robot. We expect the project to generate impact as a new approach for human-robot interaction, and to contribute novel methods for representation learning to the scientific communities in the fields of natural language understanding, machine learning, and intelligent robotics.

In this project MoReSpace, we will investigate the extent to which the transfer of learning is responsible for the development of a "self", and hypothesize that a conflict-driven attention model plays a major role. In the first part of our project, we investigate the transfer of previously learned action-effect associations to new unexpected environmental dynamics. Here, we put a strong focus on cognitive plausibility and motivate our model with psychological phenomena such as "haptic neglect". The phenomenon occurs, for example, when the computer mouse is inverted and the mouse pointer is directed in the opposite direction in each case. In such scenarios, psychologists have found reduced perception of the haptic and proprioceptive senses. Our hypothesis is that this is due to a conflict-driven attention mechanism that improves the ability to deal with such new dynamics. We will evaluate our model on a physical robot, and we will theoretically substantiate it with our collaboration partners from psychology. In the second part of the project, we will focus on imitation learning. Our hypothesis is that the attention model captures some psychological properties that are important for the human ability to change perspective and to imitate. We hypothesize that this will lead to novel methods of imitation learning for robots. We expect these methods to lead to significant improvements in the learning performance. We will evaluate this empirically and reproducibly.

In this project, we will investigate how far the transfer of learning is responsible for the development of a “self”. We will, therefore, present a computational ideomotor approach, and hypothesize that the transfer is possible due to a hierarchical structure of action-effect associations, such that training a specific narrow low-level task indirectly trains higher cognitive skills that are involved in other low-level tasks. For example, manipulating objects and balancing are two low-level tasks that both involve the cognitive skill of mental rotation. Consequently, according to our hypothesis, the mental rotation skill will benefit from balance training, which in turn also triggers an improvement of the learning of the grasping task. We will address our hypothesis by implementing a computational and neurocognitively plausible neural network architecture evaluated on a physical humanoid robot. Our expected contribution is a functional neurocognitively plausible deep reinforcement neural network model for ideomotor transfer learning that is verifiable on a reproducible robotic platform.