Multimodal Object Analysis with Auditory and Tactile Sensing Using Recurrent Neural Networks

Publikationstyp
Conference Paper
Date Issued
2021-12
Sprache
English
Author(s)
Jonetzko, Yannick  
Fiedler, Niklas  
Eppe, Manfred  
Zhang, Jianwei  
TORE-URI
http://hdl.handle.net/11420/12098
First published in
Communications in Computer and Information Science  
Number in series
1397
Start Page
253
End Page
265
Citation
5th International Conference on Cognitive Systems and Signal Processing (ICCSIP 2020)
Contribution to Conference
5th International Conference on Cognitive Systems and Signal Processing, ICCSIP 2020  
Publisher DOI
10.1007/978-981-16-2336-3_23
Scopus ID
2-s2.0-85106434963
Robots are usually equipped with many different sensors that need to be integrated. While most research is focused on the integration of vision with other senses, we successfully integrate tactile and auditory sensor data from a complex robotic system. Herein, we train and evaluate a neural network for the classification of the content of eight optically identical medicine containers. To investigate the relevance of the tactile modality in classification under realistic conditions, we apply different noise levels to the audio data. Our results show significantly higher robustness to acoustic noise with the combined multimodal network than with the unimodal audio based counterpart.
Subjects
Audio
Multimodal
Neural network
Object analysis
Tactile
MLE@TUHH