Options
Towards recognition of human actions in collaborative tasks with robots : extending action recognition with tool recognition methods
Citation Link: https://doi.org/10.15480/882.5206
Publikationstyp
Journal Article
Date Issued
2023-06-19
Sprache
English
Institut
TORE-DOI
Journal
Volume
23
Issue
12
Article Number
5718
Citation
Sensors 23 (12): 5718 (2023)
Publisher DOI
Scopus ID
Publisher
Molecular Diversity Preservation International
Peer Reviewed
true
This paper presents a novel method for online tool recognition in manual assembly processes. The goal was to develop and implement a method that can be integrated with existing Human Action Recognition (HAR) methods in collaborative tasks. We examined the state-of-the-art for progress detection in manual assembly via HAR-based methods, as well as visual tool-recognition approaches. A novel online tool-recognition pipeline for handheld tools is introduced, utilizing a two-stage approach. First, a Region Of Interest (ROI) was extracted by determining the wrist position using skeletal data. Afterward, this ROI was cropped, and the tool located within this ROI was classified. This pipeline enabled several algorithms for object recognition and demonstrated the generalizability of our approach. An extensive training dataset for tool-recognition purposes is presented, which was evaluated with two image-classification approaches. An offline pipeline evaluation was performed with twelve tool classes. Additionally, various online tests were conducted covering different aspects of this vision application, such as two assembly scenarios, unknown instances of known classes, as well as challenging backgrounds. The introduced pipeline was competitive with other approaches regarding prediction accuracy, robustness, diversity, extendability/flexibility, and online capability.
Subjects
assembly
tool detection
tool recognition
human action recognition
industrial object recognition
assembly step recognition
assembly progress detection
human–robot collaboration
DDC Class
600: Technik
620: Ingenieurwissenschaften
Funding(s)
More Funding Information
This work was supported by the German Federal Ministry of Education and Research (BMBF) under Grant Number 03HY114F within the research project H2Giga–HyPLANT100. Publishing fees supported by Funding Programme Open Access Publishing of Hamburg University of Technology (TUHH).
Loading...
Name
sensors-23-05718-v2.pdf
Size
16.95 MB
Format
Adobe PDF